From: Bob Vergeer (bob.vergeer@loyaltyworks.com)
Date: Thu Jun 09 2005 - 13:55:47 EDT
Thanks to all for the replies. Most were Oracle and Raid concerns so I
decided to simplify my task to get to the heart of it. I have a python
script (pasted below) which writes 1000 files in each of a range of
sizes then reads them back and reports on the secs to perform each task,
first buffered then unbuffered. It writes to the directory that the
command is executed from. Caveat is that it is single threaded.
All of my tests used a simple raid1 pair of 18gb disks.
System1: 1x700mhz P3 Xeon, Suse 9.0, EXT3, internal SCSI
System2: E4500 4x400mhz, Solaris 9, UFS, Internal SCSI
System3: E4500 4x400mhz, Solaris 9, UFS, Fiber attached A5200
System4: Ultra 450 1x400mhz, Solaris 9, UFS, Internal SCSI
Buffered Write
System1 System2 System3 System4
1k 0.3 1.18 6.57 12.4
2k 0.63 0.94 6.82 12.75
4k 1.27 1.02 6.43 12.94
8k 1.88 1.66 6.92 18.26
16k 1.14 2.58 14.16 24.56
32k 3.14 4.27 26.47 27.43
64k 3.21 7.73 50.44 30.07
128k 13.06 22.44 117.4 32.68
256k 13.07 43.8 263.31 43.68
Buffered Read
System1 System2 System3 System4
1k 5.5 0.3 4.42 0.26
2k 0.1 0.29 7.12 0.27
4k 0.11 0.31 3.79 0.28
8k 0.13 0.57 4.28 0.3
16k 0.18 0.47 4.42 0.47
32k 0.28 0.76 5.27 0.6
64k 0.5 1.3 16.74 1.01
128k 1.28 2.95 55.87 2.25
256k 2.4 4.63 82.67 3.76
I didn't include the unbuffered results because they were outrageous,
but more importantly, probably irrelevant.
Strictly considering system's 2&3 which are the same system with the
test conducted on internal scsi(system2) and fiber attached SCSI
(system3). I thought this would be a true apples to apples test.
The fiber results are significantly slower in both areas. I have
not been able to check cables yet as this system is off-site
but the /var/adm/messages logs are clean.
Any Thoughts?
Thanks,
Bob
P.S. Here is the python script.
##############################
# disk I/O performance measurer
import timeit, sha, time, os, shutil
fileSizes = [1, 2, 4, 8, 16, 32, 64, 128, 256] # in K bytes
numberOfFilesToWrite = 1000
workingDir = '%s/diskIOchecker/%s' % (os.getcwd(), os.getpid())
def write_buffer():
global n
n += 1
outfile = open('%s/%s_%s'%(workingDir, fileSize, n), 'w',
bufferedFlag)
outfile.write(buffer)
outfile.close()
def read_buffer():
global n
n += 1
infile = open('%s/%s_%s'%(workingDir, fileSize, n), 'r',
bufferedFlag)
infile.read()
infile.close()
def writeTest():
global totalSecs, n, fileSize, buffer
writeSecs = 0.0
print "\n%s Write Test." % buffered
print
print "File Size Seconds"
print "--------- -------"
for fileSize in fileSizes:
# build buffer containing (fileSize * 1024) random characters
buffer = ''.join([sha.new(str(x)).hexdigest()[:32] for x in
range(32*fileSize)])
#print buffer
bufferLen = len(buffer)
n = 0
timer = timeit.Timer('write_buffer()', 'from __main__ import
write_buffer')
secs = timer.timeit(numberOfFilesToWrite)
totalSecs += secs
writeSecs += secs
print "%8sK %7.2f"%(fileSize, secs)
print "\n%s Write Test completed in %.2f seconds." % (buffered,
writeSecs)
def readTest():
global totalSecs, n, fileSize, buffer
readSecs = 0.0
print "\n%s Read Test." % buffered
print
print "File Size Seconds"
print "--------- -------"
for fileSize in fileSizes:
n = 0
bufferLen = fileSize * 1024
timer = timeit.Timer('read_buffer()', 'from __main__ import
read_buffer')
secs = timer.timeit(numberOfFilesToWrite)
totalSecs += secs
readSecs += secs
print "%8sK %7.2f"%(fileSize, secs)
print "\n%s Read Test completed in %.2f seconds." % (buffered,
readSecs)
print "Number of files per file size:", numberOfFilesToWrite
print "Working Directory:", workingDir
print
totalSecs = 0.0
for buffered in 'Buffered UnBuffered'.split():
try:
os.makedirs(workingDir)
except:
pass
if buffered == 'Buffered':
bufferedFlag = -1
else:
bufferedFlag = 0
writeTest()
readTest()
shutil.rmtree(workingDir, True)
print "\nDone. Total seconds needed: %.2f" % totalSecs
##############################
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:30:52 EDT