When checking if a connection comes from localhost, don't assume it's an IPv4

When checking if a connection comes from localhost, don't assume it's an IPv4
connection.  Instead, correctly check IPv4 and IPv6 connections, assume that
AF_LOCAL is always local, and anything else is always remote.

This makes trivial local-to-remote forwarding work on my system.

[originally from svn r5180]