I/O wait over 70% for hours at a time

From: Johnson, Chad (cmjohnson@uslec.com)
Date: Wed Jul 21 2004 - 14:49:47 EDT


We have a system which, from time to time, experiances extremely high I/O
weight (>70%) for extended periods of time (sometimes hours). During this
time the amount of data transferred to the SAN is quite small (~20MB/second)
in comparision to what the SAN can handle at once (> 70MB/second). While this
IO is high we see a large number of IO operations per second.

At one time monitoring the system (only ~15% I/O wait) we saw:

Total I/O per second = 1684
Average read = ~25k
Average write = ~42k

As you can see the data throughput is rather small but the I/O ops/second is
extremely high. Is there a system limit / tunable for I/O operations per
second? While I completely understand the concept of I/O wait being a
conglomerate of factors we would still like to find a way to decrease the I/O
wait.

We do know what application creates this condition and are attempting to work
with the vendor to tune it if possible. It would appear they process many
small amounts of data at a very high rate. We are hoping we can get them to
process more data at in a single batch.

Much of the work is database work and does not touch any data external to the
database (oracle). The database is not operating on raw disk. The
filesystems are vxfs.

Your suggestions are eagerly awaited.

TIA,

Chad.
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers



This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:29:07 EDT