multihomed host - one interface works, one doesn't

From: Ian Veach (imail@nevada.edu)
Date: Sat Nov 09 2002 - 00:10:15 EST


Greetings -

I have a netra X1 with two interfaces. It is running Solaris 8 (2/02)
with recommended patches etc. and Bind 9.2.1. We are supporting a legacy
DNS IP and a new DNS IP. I was planning on rolling this out Sunday, but
seem to be having a problem:

One interface seems to resolve fine [and other net traffic works fine],
while the other does not [resolver or other net traffic] - it seems
to be related to the default gateway. That is, whichever interface is
subnet-associated with the default gateway works, and whichever is NOT
does not work. I don't understand why. Default gateway or not, shouldn't
a multihomed (but non-routing) host send traffic back out the same
interface it came in on? I'm not sure that is happening, and what's more,
snoop is returning very strange results. I've ruled out the network and bind,
and figure it must be the host and the way it is configured?

If anyone has some answers to this problem, it would be greatly
appreciated - I am rapidly running out of ideas and time. :) Here is
some information/runs from the client and netra(server) to help illustrate
the situation [the domain and network have been changed for security]:

cheers and thanks,
________________________________________________________________________
Ian 'Ivo' Veach, imail@nevada.edu UCCSN System Computing Services
http://www.nevada.edu/~ivo postmaster/webmaster/sysadmin
________________________________________________________________________

==============================================================================
Here are the interfaces on the netra. This is a multihomed host with
/etc/notrouter for a non-routing multihomed host. ip_forwarding is
obviously turned off. i also turned off ip_strict_dst_multihoming.
it supports our new DNS (migrating from Digital) and the legacy DNS - thus
the need for two addy's. Bind is on this machine.
==============================================================================

netra# ifconfig -a
lo0: flags=1000849 mtu 8232 index 1
        inet 127.0.0.1 netmask ff000000
dmfe0: flags=1000863 mtu 1500 index 2
        inet 10.10.222.10 netmask ffffff00 broadcast 10.10.222.255
        [ether removed for security]
dmfe0:1: flags=1000863 mtu 1500 index 2
        inet 10.10.222.11 netmask ffffff00 broadcast 10.10.222.255
dmfe1: flags=1000863 mtu 1500 index 3
        inet 10.10.10.134 netmask ffffff00 broadcast 10.10.10.255
        [ether removed for security]

==============================================================================
And the routing table on the netra. note the same problem occurs whether
there is a 10.10.10.134 gateway present or not.
==============================================================================

netra# netstat -nr

Routing Table: IPv4
  Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ------ ---------
10.10.10.0 10.10.10.134 U 1 10 dmfe1
10.10.10.0 10.10.10.252 UG 1 0
10.10.222.0 10.10.222.10 U 1 18 dmfe0
10.10.222.0 10.10.222.11 U 1 0 dmfe0:1
224.0.0.0 10.10.222.10 U 1 0 dmfe0
default 10.10.222.252 UG 1 44
127.0.0.1 127.0.0.1 UH 2 6 lo0

==============================================================================
Here is the client, with IP of 10.10.201.9. It points to a name server located
on the primary interface of the netra [although primary interface is not
important, what seems to be important is the default gateway on the server].
it resolves a hostname correctly. everything works so far...
(btw, the client only has one interface and one gateway).
==============================================================================

client# more resolv.conf
domain foo.com
nameserver 10.10.222.11
search foo.com
client# nslookup blackrock
Server: ns1.foo.com
Address: 10.10.222.11

Name: blackrock.foo.com
Address: 10.10.112.178

==============================================================================
Here is the traffic on the netra server. it looks ok to me, basically a DNS
request from the server running on it. I grep for the client just to keep
this to the point.
==============================================================================

netra# snoop -V -r |grep 10.10.201.9
Using device /dev/dmfe (promiscuous mode)
10.10.201.9 -> 10.10.222.11 ETHER Type=0800 (IP), size = 87 bytes
10.10.201.9 -> 10.10.222.11 IP D=10.10.222.11 S=10.10.201.9 LEN=73, ID=60273
10.10.201.9 -> 10.10.222.11 UDP D=53 S=33047 LEN=53
10.10.201.9 -> 10.10.222.11 DNS C 11.222.10.10.in-addr.arpa. Internet PTR ?
10.10.222.11 -> 10.10.201.9 ETHER Type=0800 (IP), size = 200 bytes
10.10.222.11 -> 10.10.201.9 IP D=10.10.201.9 S=10.10.222.11 LEN=186, ID=2986
10.10.222.11 -> 10.10.201.9 UDP D=33047 S=53 LEN=166
10.10.222.11 -> 10.10.201.9 DNS R 11.222.10.10.in-addr.arpa. Internet PTR ns1.foo.com.
10.10.201.9 -> 10.10.222.11 ETHER Type=0800 (IP), size = 85 bytes
10.10.201.9 -> 10.10.222.11 IP D=10.10.222.11 S=10.10.201.9 LEN=71, ID=60274
10.10.201.9 -> 10.10.222.11 UDP D=53 S=33048 LEN=51
10.10.201.9 -> 10.10.222.11 DNS C blackrock.foo.com. Internet Addr ?
10.10.222.11 -> 10.10.201.9 ETHER Type=0800 (IP), size = 181 bytes
10.10.222.11 -> 10.10.201.9 IP D=10.10.201.9 S=10.10.222.11 LEN=167, ID=2987
10.10.222.11 -> 10.10.201.9 UDP D=33048 S=53 LEN=147
10.10.222.11 -> 10.10.201.9 DNS R blackrock.foo.com. Internet Addr 10.10.112.178

==============================================================================
Now, here's where the problem occurs. Let's set the client resolver to
point to the other interface on the netra, our "legacy DNS" we have to support
for now. However, we cannot seem to resolve the name this time...
==============================================================================

client# more resolv.conf
domain foo.com
nameserver 10.10.10.134
search foo.com
client# nslookup blackrock
*** Can't find server name for address 10.10.10.134: No response from server
*** Default servers are not available

==============================================================================
Hmm. So what does the traffic look like on the server? Very strange. I
see NO initial traffic from the client. It's as if the netra somehow knows
the client is sending traffic, but why doesn't snoop see it!? It's clear
the system must see something, because these are all AckSyn's. But snoop
shows nothing, and obviously this traffic differs from above - it's obviously
a one sided conversation and I don't know why. AGAIN: If the default router
was 10.10.10.x instead of 10.10.222.x, THIS case would work but the OTHER
case would fail. So I don't think it's the network as much as this host.
==============================================================================

netra# snoop -V -r |grep 10.10.201.9
Using device /dev/dmfe (promiscuous mode)
10.10.10.134 -> 10.10.201.9 ETHER Type=0800 (IP), size = 205 bytes
10.10.10.134 -> 10.10.201.9 IP D=10.10.201.9 S=10.10.10.134 LEN=191, ID=2988
10.10.10.134 -> 10.10.201.9 UDP D=33049 S=53 LEN=171
10.10.10.134 -> 10.10.201.9 DNS R 134.10.10.10.in-addr.arpa. Internet PTR ns2new.foo.com.
10.10.10.134 -> 10.10.201.9 ETHER Type=0800 (IP), size = 205 bytes
10.10.10.134 -> 10.10.201.9 IP D=10.10.201.9 S=10.10.10.134 LEN=191, ID=2989
10.10.10.134 -> 10.10.201.9 UDP D=33049 S=53 LEN=171
10.10.10.134 -> 10.10.201.9 DNS R 134.10.10.10.in-addr.arpa. Internet PTR ns2new.foo.com.
10.10.10.134 -> 10.10.201.9 ETHER Type=0800 (IP), size = 205 bytes
10.10.10.134 -> 10.10.201.9 IP D=10.10.201.9 S=10.10.10.134 LEN=191, ID=2990
10.10.10.134 -> 10.10.201.9 UDP D=33049 S=53 LEN=171
10.10.10.134 -> 10.10.201.9 DNS R 134.10.10.10.in-addr.arpa. Internet PTR ns2new.foo.com.
10.10.10.134 -> 10.10.201.9 ETHER Type=0800 (IP), size = 205 bytes
10.10.10.134 -> 10.10.201.9 IP D=10.10.201.9 S=10.10.10.134 LEN=191, ID=2991
10.10.10.134 -> 10.10.201.9 UDP D=33049 S=53 LEN=171
10.10.10.134 -> 10.10.201.9 DNS R 134.10.10.10.in-addr.arpa. Internet PTR ns2new.foo.com.

=== fini
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers



This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:25:15 EDT