From - Fri Sep 24 02:34:26 1999
Return-path: <dclarke@interlog.com>
Envelope-to: dclarke@interlog.com
Delivery-date: Tue, 15 Jun 1999 01:07:22 -0400
Received: from smtp.interlog.com ([207.34.202.37])
        by mailhub2.interlog.com with esmtp (Exim 2.05 #1)
        id 10tlRS-0005K1-00
        for dclarke@interlog.com; Tue, 15 Jun 1999 01:07:22 -0400
Received: from interlog.com (dclarke.interlog.com [204.225.242.227])
        by smtp.interlog.com (8.9.1/8.9.1) with ESMTP id BAA10054
        for ; Tue, 15 Jun 1999 01:07:57 -0400 (EDT)
Sender: dclarke@interlog.com
Message-ID: <3765A667.3E745BF4@interlog.com>
Date: Tue, 15 Jun 1999 06:03:35 +0500
From: Dennis Clarke <dclarke@interlog.com>
X-Mailer: Mozilla 4.5 [en] (X11; I; SunOS 5.5.1 i86pc)
X-Accept-Language: en
MIME-Version: 1.0
Newsgroups: comp.unix.solaris
Subject: DiskSuite Step by Step ...
References: <3765B041.967D7EF7@mediaone.net>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Bcc:
X-SPAM-Warning: unqualified To: line
X-Mozilla-Status: 9001
X-Mozilla-Status2: 00000000
X-UIDL: 554043dabef0a630ef167be42fb53ec5

Whenever I see a message about disksuite frustrations, I feel compelled
to write. Thus, here I am. DiskSuite is one of my favorite products
and I mirror everything I can.

First, you task of mirroring two drives with DiskSuite can be done in
under 30 minutes by a seasoned DiskSuite type, be of good cheer, you're
doing fine. Don't expect to see it as being obvious for a while. For
all intents and purposes, think of a metadevice as any other disk type
device. For the most part, you need not know the nitty gritty bits
about how data really gets to the surface of your hard-disk do you? It
would take years of study in materials physics and computer programming
to really get it. I don't. The metadevice is like a 'magic' disk drive
that just happens to distribute data reads and writes to a set of
submirrors. It is a driver based interface to your actual hardware and
all you really need to know to get things working is HOW to get it
working, not WHY it is working. Once it is up and running for you, then
you can read the manuals to your hearts content, but for now, let's just
make it work.

Step 1) Install Solstice DiskSuite properly.

OK, well, you happen to be running Solaris 7 for x86. I have Solaris 7
for Sparc here as well as Solaris 2.5.1 for x86 ( this workstation ) and
both are using DiskSuite ( see above - I mirror everything ) and the
word "everything" is pronounced with a roll of the head and a wave of
the hands for dramatic effect. You have to read carefully to see that.

Follow the standard package installation process to get SUNWmd
installed. When done, you should be able to check it thus:


$ pkginfo -l SUNWmd
   PKGINST:  SUNWmd
      NAME:  Solstice DiskSuite
  CATEGORY:  system
      ARCH:  all
   VERSION:  4.0,REV=1.0
   BASEDIR:  /
    VENDOR:  SunSoft, a Sun Microsystems, Inc. Business
      DESC:  SunSoft's Solstice DiskSuite
    PSTAMP:  airjazz950414160211
  INSTDATE:  Dec 11 1998 13:40
    VSTOCK:  258-4903-10
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:     75 installed pathnames
                41 shared pathnames
                22 directories
                23 executables
              6948 blocks used (approx)


The above is DiskSuite Rev 4.0 for Solaris 2.5.1 on Intel. The Sun Sparc 20
hardware says the following :


$ pkginfo -l SUNWmd
   PKGINST:  SUNWmd
      NAME:  Solstice DiskSuite
  CATEGORY:  system
      ARCH:  sparc
   VERSION:  4.2,REV=1998.02.09.12.47.28
   BASEDIR:  /
    VENDOR:  Sun Microsystems, Inc.
      DESC:  Solstice DiskSuite
    PSTAMP:  09/02/98-13:04:05
  INSTDATE:  May 29 1999 00:08
    VSTOCK:  258-6252-11
   HOTLINE:  Please contact your local service provider
    STATUS:  completely installed
     FILES:     95 installed pathnames
                22 shared pathnames
                25 directories
                37 executables
             13496 blocks used (approx)


I have no idea why the ARCH tag says 'all' for x86 on 2.5.1 and is
correct with 'sparc' for Solaris 7 on Sparc. Let's not worry about it,
some Sun SE can answer that sort of minutia.

Step 2) Set up your drive slices :^)


The smile will tell you that you had better see DiskSuite being in your
plan from the beginning because its a bit of a pain if you need to free
up a slice for the metadbs. Without going to the manual, here are the
next steps ...

Suppose that you have the following disks ...

Example 1) Sun Sparc 20 with dual drives ... 


# format
Searching for disks...done
 
 
AVAILABLE DISK SELECTIONS:
       0. c0t1d0 
          /iommu@f,e0000000/sbus@f,e0001000/espdma@f,400000/esp@f,800000/sd@1,0
       1. c0t3d0 
          /iommu@f,e0000000/sbus@f,e0001000/espdma@f,400000/esp@f,800000/sd@3,0
Specify disk (enter its number):


Then it seems reasonable to mirror these identical drives. But you
don't need to have identical drives! You can mirror a 256Mb slice with
a 1Gb slice if you want, I don't recommend it. DiskSuite will only
mirror 256Mb of the filesystem and ignore the wasted area of the misfit
second submirror slice. For the sake of seeing a real hack, here is my
x86 workstation made from whatever drives were kicking about at the time.




# format
Searching for disks...done
 
 
AVAILABLE DISK SELECTIONS:
       0. c0t0d0 
          /pci@0,0/pci1000,1@1/cmdk@0,0
       1. c0t2d0 
          /pci@0,0/pci1000,1@1/cmdk@2,0
       2. c0t3d0 
          /pci@0,0/pci1000,1@1/cmdk@3,0
Specify disk (enter its number):


Take note that these drives are a real miss-mash of whatever. With
DiskSuite, it doesn't matter that the drive geometry is different (
actually, for performance, it does, but lets not split hairs for now )
you can mirror any thing with almost anything. You can concatenate
anything with anything. Just don't make your submirrors or
sub-components of a concat on the same drive! That will cause incredible
drive thrashing for obvious reasons.

OK, so here is the partition layout of the Sparc20 :
Target 1 drive ... 


format> curr
Current Disk = c0t1d0

/iommu@f,e0000000/sbus@f,e0001000/espdma@f,400000/esp@f,800000/sd@1,0

Current partition table (original):
Total disk cylinders available: 3880 + 2 (reserved cylinders)
 
Part      Tag    Flag     Cylinders        Size            Blocks
  0       root    wm       0 -  121      128.67MB    (122/0/0)   263520
  1        var    wm     122 -  364      256.29MB    (243/0/0)   524880
  2     backup    wu       0 - 3879        4.00GB    (3880/0/0) 8380800
  3       swap    wu     365 -  607      256.29MB    (243/0/0)   524880
  4 unassigned    wm     608 -  617       10.55MB    (10/0/0)     21600
  5 unassigned    wm     618 - 2559        2.00GB    (1942/0/0) 4194720
  6        usr    wm    2560 - 3818        1.30GB    (1259/0/0) 2719440
  7 unassigned    wm    3819 - 3879       64.34MB    (61/0/0)    131760


Target 3 drive ... 


format> curr
Current Disk = c0t3d0

/iommu@f,e0000000/sbus@f,e0001000/espdma@f,400000/esp@f,800000/sd@3,0

Current partition table (original):
Total disk cylinders available: 3880 + 2 (reserved cylinders)
 
Part      Tag    Flag     Cylinders        Size            Blocks
  0       root    wm       0 -  121      128.67MB    (122/0/0)   263520
  1        var    wm     122 -  364      256.29MB    (243/0/0)   524880
  2     backup    wm       0 - 3879        4.00GB    (3880/0/0) 8380800
  3       swap    wu     365 -  607      256.29MB    (243/0/0)   524880
  4 unassigned    wm     608 -  617       10.55MB    (10/0/0)     21600
  5 unassigned    wm     618 - 2559        2.00GB    (1942/0/0) 4194720
  6        usr    wm    2560 - 3818        1.30GB    (1259/0/0) 2719440
  7 unassigned    wm    3819 - 3879       64.34MB    (61/0/0)    131760


PAY PARTICULAR ATTENTION TO SLICE s4 on these identical drives ...

That tiny little slice is where the metadb replica databases are going.

My x86 system is similar but for the moment lets stick with a sane
system that will have identical drives like yours.

Step 3) Create the metadb replica databases ...


Because you must have at least two metadb replicas available for the
system to work, I suggect that your chosen slice have two metadb
replicas in it, on each drive. That way you have four metadbs in total
and you can lose either drive and keep running.

su to root and set up your path for sanity ...


PATH=/usr/sbin:/usr/bin:/usr/opt/SUNWmd/sbin;export PATH

and then create the metadbs in /dev/dsk/c0t1d0s4 and /dev/dsk/c0t3d0s4
and make sure your boot drive has the first metadb replica db ...




# metadb -a -c 2 -f /dev/dsk/c0t3d0s4 /dev/dsk/c0t1d0s4

and that creates the four needed dbs as ( note "-c 2" ) 2 dbs per slice.

Check your /etc/system file and you will see some new stuff at the
bottom :


* Begin MDD database info (do not edit)
set md:mddb_bootlist1="sd:28:16 sd:28:1050 sd:12:16 sd:12:1050"
* End MDD database info (do not edit)

At this point, if you want, you can reboot. You will see a stack of
WARNING messages, not errors, WARNINGS only ( hand waving again ) about
a stack of failures due to raid this and trans that not loading. Ignore
this. Everyone has fits when they see it thinking it means some
catastrophe has happened to their system. Not yet. :) If you use
metadb -i you should see


# metadb -i
        flags           first blk       block count
     a m  p  luo        16              1034           /dev/dsk/c0t3d0s4
     a    p  luo        1050            1034           /dev/dsk/c0t3d0s4
     a    p  luo        16              1034           /dev/dsk/c0t1d0s4
     a    p  luo        1050            1034           /dev/dsk/c0t1d0s4
 o - replica active prior to last mddb configuration change
 u - replica is up to date
 l - locator for this replica was read successfully
 c - replica's location was in /etc/opt/SUNWmd/mddb.cf
 p - replica's location was patched in kernel
 m - replica is master, this is replica selected as input
 W - replica has device write errors
 a - replica is active, commits are occurring to this replica
 M - replica had problem with master blocks
 D - replica had problem with data blocks
 F - replica had format problems
 S - replica is too small to hold current data base
 R - replica had device read errors
#

Step 4) Mirror your root filesystem.

Figure out where your root submirror slices are. Hopefully they are s0
on both drives.

Edit the file /etc/opt/SUNWmd/md.tab and put in the following ...


# root mirror 
d10 1 1 /dev/dsk/c0t3d0s0
d0 -m d10
d20 1 1 /dev/dsk/c0t1d0s0

And then save this file. Don't worry that the manual says you need to
type in /dev/md/dsk/d10 in the place of d10, you don't need it.
Solstice DiskSuite is a really smart product and it generally knows that
d10 is a metadevice. So keep is short and sweet in the md.tab file.

[ This section is specific to x86 architecture - verify with the manual ]

Make sure that the second submirror slice is a bootable filesystem.


# installboot /usr/lib/fs/ufs/pboot /usr/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0

Now initialize the mirror d0 


# metainit d20
concat/stripe setup

# metainit -f d10
concat/stripe setup

# metainit d0
mirror setup

...........and finally 


# metaroot d0

If you check your /etc/vfstab file you will see a few changes ...
Reboot with your newly mirror root filesystem.

Step 5) Mirror the rest of the slices.

edit /etc/opt/SUNWmd/md.tab and add the following ( or whatever you want ) 


# swap mirror
d11 1 1 /dev/dsk/c0t3d0s3
d1 -m d11
d21 1 1 /dev/dsk/c0t1d0s3
 
# usr mirror
d12 1 1 /dev/dsk/c0t3d0s6
d2 -m d12
d22 1 1 /dev/dsk/c0t1d0s6
 
# var mirror
d13 1 1 /dev/dsk/c0t3d0s1
d3 -m d13
d23 1 1 /dev/dsk/c0t1d0s1

# opt mirror
d15 1 1 /dev/dsk/c0t3d0s5
d5 -m d15
d25 1 1 /dev/dsk/c0t1d0s5

and then init all that like so ... 


# metainit d21
concat/stripe setup

# metainit -f d11
concat/stripe setup

# metainit d1
mirror setup

see a pattern here? Always init the inactive slice first. Then the
active slice. Then the actual mirror device. Ensure that the mirror
device ( like d1 ) has the ACTIVE slice attached. If you do it the
other way ( any other way) then plan for a disaster! You would end up
mirroring a blank slice with an active slice ... not pretty.

Edit your /etc/vfstab ... ensure that the new metadbs are being mounted and
fsck'ed ...



# cat /etc/vfstab
#device         device          mount           FS      fsck    mount  mount
#to mount       to fsck         point           type    pass    at boot options
#
#/dev/dsk/c1d0s2 /dev/rdsk/c1d0s2 /usr          ufs     1       yes    -
fd      -       /dev/fd fd      -       no      -
/proc   -       /proc   proc    -       no      -
/dev/md/dsk/d1  -       -       swap    -       no      -
/dev/md/dsk/d0  /dev/md/rdsk/d0 /       ufs     1       no      -
/dev/md/dsk/d2  /dev/md/rdsk/d2 /usr    ufs     1       no      -
/dev/md/dsk/d3  /dev/md/rdsk/d3 /var    ufs     1       no      -
/dev/md/dsk/d5  /dev/md/rdsk/d5 /opt    ufs     2       yes     -
swap    -       /tmp    tmpfs   -       yes     -

If you're wondering what happened to d4 ... that's another topic ( ufs
logging )

Reboot and you will have a system that is kinda mirrored. You now have
to attach the second submirror device to all those one way mirrors.


metattach d0 d20
metattach d1 d21
metattach d2 d22
.
.
.
etc
.
.
.

And you will hear a lot of drive thrashing.

That's about it.


Dennis Clarke
dclarke@interlog.com
Certified Solaris Administrator
Certified Lotus Notes Professional





Go to Livewire
Email >dclarke@interlog.com