From: ramesh kurup (rameshkurupster@gmail.com)
Date: Fri Dec 08 2006 - 08:54:39 EST
Dear Sunmanagers
Here we are having 2 SUN Fire V890 server are configured under
SUN Cluster 3.1. Two days before the database(Oracle) has shutdown.On
the same time i found some errors regarding to the failure of ipmp0
and ipmp1 of both servers. Here i have pasted the error messages.
Please provide me some solutions.
Hostname of the servers are dbsrv1 and dbsrv2
dbsrv1 (1st server on cluster)
/var/adm/messages
Dec 6 11:29:31 estamp-dbsrv1 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:02' trying to be our address
172.016.193.002!
Dec 6 12:01:53 estamp-dbsrv1 in.mpathd[112]: [ID 168056 daemon.error]
All Interfaces in group ipmp1 have failed
Dec 6 12:01:53 estamp-dbsrv1 Cluster.PNM: [ID 890413 daemon.notice]
ipmp1: state transition from OK to DOWN.
Dec 6 12:01:54 estamp-dbsrv1 in.mpathd[112]: [ID 168056 daemon.error]
All Interfaces in group ipmp1 have failed
Dec 6 12:09:31 estamp-dbsrv1 ip: [ID 903730 kern.warning] WARNING:
- Ignored:
IP: Hardware address '00:00:00:00:00:02' trying to be our address
172.016.193.002!
Dec 6 12:14:14 estamp-dbsrv1 in.mpathd[112]: [ID 299542 daemon.error]
NIC repair detected on ce0 of group ipmp1
Dec 6 12:14:14 estamp-dbsrv1 in.mpathd[112]: [ID 620804 daemon.error]
Successfully failed back to NIC ce0
Dec 6 12:14:14 estamp-dbsrv1 in.mpathd[112]: [ID 237757 daemon.error]
At least 1 interface (ce0) of group ipmp1 has repaired
Dec 6 12:14:14 estamp-dbsrv1 Cluster.PNM: [ID 890413 daemon.notice]
ipmp1: state transition from DOWN to OK.
Dec 6 12:14:16 estamp-dbsrv1 in.mpathd[112]: [ID 299542 daemon.error]
NIC repair detected on ce1 of group ipmp1
Dec 6 12:14:16 estamp-dbsrv1 in.mpathd[112]: [ID 620804 daemon.error]
Successfully failed back to NIC ce1
Dec 6 12:49:31 estamp-dbsrv1 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:02' trying to be our address
172.016.193.002!
ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
inet 127.0.0.1 netmask ff000000
ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.112.101 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp1
ether 0:14:4f:24:f8:54
ce0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 2
inet 192.168.112.102 netmask ffffff00 broadcast 192.168.112.255
ce1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3
inet 192.168.112.103 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp1
ether 0:14:4f:24:f8:55
ce3: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu
1500 index 5
inet 172.16.0.130 netmask ffffff80 broadcast 172.16.0.255
ether 0:14:4f:24:f8:57
eri0: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu
1500 index 4
inet 172.16.1.2 netmask ffffff80 broadcast 172.16.1.127
ether 0:14:4f:23:b5:44
clprivnet0: flags=1009843<UP,BROADCAST,RUNNING,MULTICAST,MULTI_BCAST,PRIVATE,IPv4>
mtu 1500 index 6
inet 172.16.193.2 netmask ffffff00 broadcast 172.16.193.255
ether 0:0:0:0:0:2
/etc/hosts
#
# Internet host table
#
127.0.0.1 localhost
192.168.112.101 estamp-dbsrv1 loghost node1
192.168.112.111 estamp-dbsrv2 node2
###Added For Cluster####
192.168.112.102 estamp-dbsrv1-test-ce0
192.168.112.103 estamp-dbsrv1-test-ce1
###Logical Hostname For Cluster####
192.168.112.114 estamp-dbsrv1-vip
192.168.112.115 estamp-dbsrv2-vip
172.16.193.2 estamp-dbsrv1-priv
172.16.193.1 estamp-dbsrv2-priv
dbsrv2 (IInd server on cluster)
/var/adm/messages
Dec 6 12:01:53 estamp-dbsrv2 in.mpathd[110]: [ID 168056 daemon.error]
All Interfaces in group ipmp0 have failed
Dec 6 12:01:53 estamp-dbsrv2 Cluster.PNM: [ID 890413 daemon.notice]
ipmp0: state transition from OK to DOWN.
Dec 6 12:01:54 estamp-dbsrv2 in.mpathd[110]: [ID 168056 daemon.error]
All Interfaces in group ipmp0 have failed
Dec 6 12:14:15 estamp-dbsrv2 in.mpathd[110]: [ID 299542 daemon.error]
NIC repair detected on ce1 of group ipmp0
Dec 6 12:14:15 estamp-dbsrv2 in.mpathd[110]: [ID 620804 daemon.error]
Successfully failed back to NIC ce1
Dec 6 12:14:15 estamp-dbsrv2 in.mpathd[110]: [ID 237757 daemon.error]
At least 1 interface (ce1) of group ipmp0 has repaired
Dec 6 12:14:15 estamp-dbsrv2 Cluster.PNM: [ID 890413 daemon.notice]
ipmp0: state transition from DOWN to OK.
Dec 6 12:14:16 estamp-dbsrv2 in.mpathd[110]: [ID 299542 daemon.error]
NIC repair detected on ce0 of group ipmp0
Dec 6 12:14:16 estamp-dbsrv2 in.mpathd[110]: [ID 620804 daemon.error]
Successfully failed back to NIC ce0
Dec 6 16:10:14 estamp-dbsrv2 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:01' trying to be our address
172.016.193.001!
ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
inet 127.0.0.1 netmask ff000000
ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.112.111 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp0
ether 0:14:4f:24:f8:70
ce0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 2
inet 192.168.112.112 netmask ffffff00 broadcast 192.168.112.255
ce1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3
inet 192.168.112.113 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp0
ether 0:14:4f:24:f8:71
ce3: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu
1500 index 5
inet 172.16.0.129 netmask ffffff80 broadcast 172.16.0.255
ether 0:14:4f:24:f8:73
eri0: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu
1500 index 4
inet 172.16.1.1 netmask ffffff80 broadcast 172.16.1.127
ether 0:14:4f:1e:46:f
clprivnet0: flags=1009843<UP,BROADCAST,RUNNING,MULTICAST,MULTI_BCAST,PRIVATE,IPv4>
mtu 1500 index 6
inet 172.16.193.1 netmask ffffff00 broadcast 172.16.193.255
ether 0:0:0:0:0:1
/etc/hosts
#
# Internet host table
#
127.0.0.1 localhost
192.168.112.101 estamp-dbsrv1 loghost node1
192.168.112.111 estamp-dbsrv2 node2
###Added For Cluster####
192.168.112.102 estamp-dbsrv1-test-ce0
192.168.112.103 estamp-dbsrv1-test-ce1
###Logical Hostname For Cluster####
192.168.112.114 estamp-dbsrv1-vip
192.168.112.115 estamp-dbsrv2-vip
172.16.193.2 estamp-dbsrv1-priv
172.16.193.1 estamp-dbsrv2-priv
Advance Thanks
Ramesh R S Kurup
- Done.
---------- Forwarded message ----------
From: "ramesh kurup" <rameshkurupster@gmail.com>
To: sunmanagers-request@sunmanagers.org
Date: Fri, 8 Dec 2006 18:50:50 +0530
Subject: IPMP fail and ip address problem
Hi sunmanagers,
Here we are having 2 SUN Fire V890 server are configured under
SUN Cluster 3.1. Two days before the database(Oracle) has shutdown.On
the same time i found some errors regarding to the failure of ipmp0
and ipmp1 of both servers. Here i have pasted the error messages.
Please provide me some solutions.
Hostname of the servers are dbsrv1 and dbsrv2
dbsrv1 (1st server on cluster)
/var/adm/messages
Dec 6 11:29:31 estamp-dbsrv1 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:02' trying to be our address
172.016.193.002!
Dec 6 12:01:53 estamp-dbsrv1 in.mpathd[112]: [ID 168056 daemon.error]
All Interfaces in group ipmp1 have failed
Dec 6 12:01:53 estamp-dbsrv1 Cluster.PNM: [ID 890413 daemon.notice]
ipmp1: state transition from OK to DOWN.
Dec 6 12:01:54 estamp-dbsrv1 in.mpathd[112]: [ID 168056 daemon.error]
All Interfaces in group ipmp1 have failed
Dec 6 12:09:31 estamp-dbsrv1 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:02' trying to be our address
172.016.193.002!
Dec 6 12:14:14 estamp-dbsrv1 in.mpathd[112]: [ID 299542 daemon.error]
NIC repair detected on ce0 of group ipmp1
Dec 6 12:14:14 estamp-dbsrv1 in.mpathd[112]: [ID 620804 daemon.error]
Successfully failed back to NIC ce0
Dec 6 12:14:14 estamp-dbsrv1 in.mpathd[112]: [ID 237757 daemon.error]
At least 1 interface (ce0) of group ipmp1 has repaired
Dec 6 12:14:14 estamp-dbsrv1 Cluster.PNM: [ID 890413 daemon.notice]
ipmp1: state transition from DOWN to OK.
Dec 6 12:14:16 estamp-dbsrv1 in.mpathd[112]: [ID 299542 daemon.error]
NIC repair detected on ce1 of group ipmp1
Dec 6 12:14:16 estamp-dbsrv1 in.mpathd[112]: [ID 620804 daemon.error]
Successfully failed back to NIC ce1
Dec 6 12:49:31 estamp-dbsrv1 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:02' trying to be our address
172.016.193.002!
ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
inet 127.0.0.1 netmask ff000000
ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.112.101 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp1
ether 0:14:4f:24:f8:54
ce0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 2
inet 192.168.112.102 netmask ffffff00 broadcast 192.168.112.255
ce1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3
inet 192.168.112.103 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp1
ether 0:14:4f:24:f8:55
ce3: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 5
inet 172.16.0.130 netmask ffffff80 broadcast 172.16.0.255
ether 0:14:4f:24:f8:57
eri0: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu
1500 index 4
inet 172.16.1.2 netmask ffffff80 broadcast 172.16.1.127
ether 0:14:4f:23:b5:44
clprivnet0: flags=1009843<UP,BROADCAST,RUNNING,MULTICAST,MULTI_BCAST,PRIVATE,IPv4>
mtu 1500 index 6
inet 172.16.193.2 netmask ffffff00 broadcast 172.16.193.255
ether 0:0:0:0:0:2
/etc/hosts
#
# Internet host table
#
127.0.0.1 localhost
192.168.112.101 estamp-dbsrv1 loghost node1
192.168.112.111 estamp-dbsrv2 node2
###Added For Cluster####
192.168.112.102 estamp-dbsrv1-test-ce0
192.168.112.103 estamp-dbsrv1-test-ce1
###Logical Hostname For Cluster####
192.168.112.114 estamp-dbsrv1-vip
192.168.112.115 estamp-dbsrv2-vip
172.16.193.2 estamp-dbsrv1-priv
172.16.193.1 estamp-dbsrv2-priv
dbsrv2 (IInd server on cluster)
/var/adm/messages
Dec 6 12:01:53 estamp-dbsrv2 in.mpathd[110]: [ID 168056 daemon.error]
All Interfaces in group ipmp0 have failed
Dec 6 12:01:53 estamp-dbsrv2 Cluster.PNM: [ID 890413 daemon.notice]
ipmp0: state transition from OK to DOWN.
Dec 6 12:01:54 estamp-dbsrv2 in.mpathd[110]: [ID 168056 daemon.error]
All Interfaces in group ipmp0 have failed
Dec 6 12:14:15 estamp-dbsrv2 in.mpathd[110]: [ID 299542 daemon.error]
NIC repair detected on ce1 of group ipmp0
Dec 6 12:14:15 estamp-dbsrv2 in.mpathd[110]: [ID 620804 daemon.error]
Successfully failed back to NIC ce1
Dec 6 12:14:15 estamp-dbsrv2 in.mpathd[110]: [ID 237757 daemon.error]
At least 1 interface (ce1) of group ipmp0 has repaired
Dec 6 12:14:15 estamp-dbsrv2 Cluster.PNM: [ID 890413 daemon.notice]
ipmp0: state transition from DOWN to OK.
Dec 6 12:14:16 estamp-dbsrv2 in.mpathd[110]: [ID 299542 daemon.error]
NIC repair detected on ce0 of group ipmp0
Dec 6 12:14:16 estamp-dbsrv2 in.mpathd[110]: [ID 620804 daemon.error]
Successfully failed back to NIC ce0
Dec 6 16:10:14 estamp-dbsrv2 ip: [ID 903730 kern.warning] WARNING:
IP: Hardware address '00:00:00:00:00:01' trying to be our address
172.016.193.001!
ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
inet 127.0.0.1 netmask ff000000
ce0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.112.111 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp0
ether 0:14:4f:24:f8:70
ce0:1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 2
inet 192.168.112.112 netmask ffffff00 broadcast 192.168.112.255
ce1: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER>
mtu 1500 index 3
inet 192.168.112.113 netmask ffffff00 broadcast 192.168.112.255
groupname ipmp0
ether 0:14:4f:24:f8:71
ce3: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu 1500 index 5
inet 172.16.0.129 netmask ffffff80 broadcast 172.16.0.255
ether 0:14:4f:24:f8:73
eri0: flags=1008843<UP,BROADCAST,RUNNING,MULTICAST,PRIVATE,IPv4> mtu
1500 index 4
inet 172.16.1.1 netmask ffffff80 broadcast 172.16.1.127
ether 0:14:4f:1e:46:f
clprivnet0: flags=1009843<UP,BROADCAST,RUNNING,MULTICAST,MULTI_BCAST,PRIVATE,IPv4>
mtu 1500 index 6
inet 172.16.193.1 netmask ffffff00 broadcast 172.16.193.255
ether 0:0:0:0:0:1
/etc/hosts
#
# Internet host table
#
127.0.0.1 localhost
192.168.112.101 estamp-dbsrv1 loghost node1
192.168.112.111 estamp-dbsrv2 node2
###Added For Cluster####
192.168.112.102 estamp-dbsrv1-test-ce0
192.168.112.103 estamp-dbsrv1-test-ce1
###Logical Hostname For Cluster####
192.168.112.114 estamp-dbsrv1-vip
192.168.112.115 estamp-dbsrv2-vip
172.16.193.2 estamp-dbsrv1-priv
172.16.193.1 estamp-dbsrv2-priv
Advance Thanks
Ramesh R S Kurup
-- ----- Fight back spam! Download the Blue Frog. http://www.bluesecurity.com/register/s?user=cmFtZXNoa3VydXBzdGVy _______________________________________________ sunmanagers mailing list sunmanagers@sunmanagers.org http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:41:18 EDT