TruCluster 5.1A -> 5.1B rolling upgrade fun and games

From: Tim Cutts (tjrc@sanger.ac.uk)
Date: Wed Dec 17 2003 - 18:12:07 EST


We have several ES45 clusters here, most of which we have already
upgraded to 5.1B. I've done two such upgrades myself, with minimal
problems (one 8 node cluster, one 4 node cluster).

Until today.

System: 6 node quad 1GHz ES45 cluster, memory channel interconnect

The upgrade was going fine until I started the installupdate on lead
node. It announced, perfectly normally, that both Tru64 and TruCluster
would be upgraded to the new 5.1B versions. It installed the 102
packages of Tru64. Fine. It merged a couple of proto files. Fine. It
then decided to rebuild its kernel and reboot, without having installed
the TruCluster packages.

Unsurprisingly, the resulting reboot caused the machine to panic
(fortunately it didn't take the rest of the cluster with it).

Booting genvmunix and poking around with some help from HP confirmed
that the TruCluster packages had not been installed, and the resulting
kernel that had been built did not have any TruCluster awareness. This
caused the panic since it couldn't talk to the clustered disks.

I've tried this three times now, in slightly different ways (RIS, local
installation media etc etc) and have now (fortunately) successfully
backed out the process to the original 5.1A level.

Has anyone seen this happen before, and know what the cause is? I'll
call it in with HP in the morning, but the levels of wisdom available
through this list might help us out...

Thanks!

Tim

-- 
Dr Tim Cutts
Informatics Systems Group
Wellcome Trust Sanger Institute, Hinxton, Cambridge, CB10 1SA, UK


This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:49:46 EDT