From: Dave Markham (dave.markham@fjserv.net)
Date: Fri May 05 2006 - 06:30:31 EDT
This really is a last resort but im having a multipathing
problem with a new 4500 and t3 installation we have which is trying to
mimic one of the environments we have elsewhere.
Setup =
2 * E4500's in a cluster environment with sun cluster 3.0
2 hba's QLC in each node (x6757a fibre sbus)
2 sanbox8 switches
2 t3+ in a partner pair configuration.
One hba on one node goes to one switch and the second hba goes to the second switch.
The second node is reversed so hba1 goes to second switch and hba2 goes to first switch.
One t3 brick is plugged into each switch.
Basically we have dd'd disks of 2 cluster nodes at our live site and rebuilt 2 new
4500's using the dd images.
They are clustered also.
Both nodes are identical but i shall talk about just one.
I have mp_support enabled on the t3 and the switches configured as i see
them configured on the live site so the ports ( sanbox 8 switches) are
set to G port mode on the gui and i can see they are green with an F in
which i assume is fibre mode so to me they are ok.
Now this is what i have done on the nodes to try and get multipathing
working.
We had various problems with did devices although were trying to boot -x
in none cluster mode. We have resolved that by clearing them out. Below
is process of what i have done starting off with clearing things and not
enabling multipathing to try and start afresh.
c6 and c8 are the channels for the fibre hbas we have on each node
cfgadm -c unconfigure c6 c8
devfsadm -C
scdidadm -C
change /kernel/drv/scsi_vhci.conf so it disables mpxio ( set to yes )
then took box down and boot -r ( we tried various times of booting -rx
but it bums out and says wont update path_to_inst as not in cluster mode )
So now should have system with nothing and no mpxio. So....
Enable mp_support mpxio on the t3's ( sys mp_support mpxio )
cfgadm -c configure c6 c8 on the node so it can now see all luns through
both paths. ( there are 5 slices on each t3 so through multipathing = 10 )
scdidadm -r to build the did devices
devfsadm -C to clear any old things out
changed scsi_vhci.conf to enable mpxio and then took down and boot -r
I am expecting now in format to see the mpxio place holder and it to
have created devices in /dev/rdsk and /devices/scsi_vhci for a C10
device and a long path name which if i did a luxadm display on would
show 2 paths to the disks.
Unfortunately this hasn't happened and i still just see c6 and c8 disks
under format and the directory /devices/scsi_vhci isnt there. I am now
stuck.
I have also tried not enabling mp_support on the t3s so it just sees 5
luns from each c6 and c8 on the node ( as apposed to the 10 it provides
if mp_support is set to mpxio ( due to 5 slices/luns on each t3 )).
Have you ever seen anything like this or have any tip as to what to try.
All docs i can find say to do what i have done and it should work. The
only thing im worried about is as all the patches and packages are
installed already ( from the dd images ) for san stuff does anything
need to be re-installed. I have thought it should not do as what happens
in the situation if someones t3 partner pair blows up and they get whole
new devices. They would be in similar situation to us and need to
rebuild the device paths etc.
Any info or points in direction would be massive help. As usual stupid project managers want a conference call daily for progress like that helps.
Cheers
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:39:47 EDT