From: Mark Wiederspahn (markw@utig.ig.utexas.edu)
Date: Wed May 12 2004 - 17:43:56 EDT
FC router, scsi tape, poor performance
I have a 280R, solaris 8, up to date patches, with one channel
of a Sun dual 1Gb qlogic 2200 hba going to a Chapparal vfs226
fc-scsi router (test configuration). A tape library and a standalone
tape drive and some scsi disks are on on its 6 lvd-160 scsi channels.
These are AIT-3 and Exabyte M2 drives, Qualstar library.
San and multi-pathing software/patches are installed.
My question is about unusual performance limits. The
second half of the question is the interesting part:
If I attempt to saturate the channel with io, in a simple form,
dd if=/dev/zero bs=128k of=some-tape, I see 40MB/s to
a single drive. If I write to two drives, on separate scsi buses,
I see 2x22 MB/s. If I write to N drives, I see roughly 40/N
MB/s per drive. This is much lower than one would expect from a
1G Fibre channel, unless there were a -lot- of contention.
If however, I do even a bit of disk traffic to a scsi disk on
a different scsi bus on the fc router, I see 2x40 MB/s, up to
a total bandwidth of 96MB/s writing, or about 85MB/s reading.
(eg, 3x32MB/s, 4x24MB/s, and so forth). Originally noticed
using fsck, but tested with a C program with adjustable nanosleep();
the effect is immediate with even one raw disk read (of 512 bytes)
per second. The tape rates slow down within seconds of the disk
io traffic being stopped, and immediately increase when it is
started. iostat -x 5 is used to monitor the traffic.
Single stream results don't vary with disk traffic. Larger block sizes
(eg 256k) show faster rates (46MB/s), but the same effect. The effect has
been seen with the actual application (samfs) with "real" data,
with about 30-50% non-compressible data rate improvement.
Library vendor claims this is a Solaris platform oddity,
and recommended scsi bus hardware loggers to debug it.
The observed disk effect makes me believe the claim, and
because another user saw something similar.
forum.sun.com in the Storedge forum, has a similar post from about
a year ago with SDLT320 drives and 2G Sun HBA's and a Crossroads
router. He found that using a second FC port essentially removed
the 40/N limit. Another poster gave no details, but recently said
"me too". Search on "crossroads" for the thread.
thanks,
mw
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:28:38 EDT