High IO-Wait at V480 server with Sun Solaris 9 on - how to fix

From: ken lee (kennethkklee@gmail.com)
Date: Thu Nov 02 2006 - 10:48:22 EST


Hi all,

Looking for a solution to solve High IO Wait (around 85%) issue at V480 with
Sun Solaris 9 on. Some statistic (Top, iostat) are listed as below.
Thanks in advance.

Top shows:

last pid: 25228; load averages: 0.19, 0.22, 0.25
15:32:37
115 processes: 114 sleeping, 1 on cpu
CPU states: 6.6% idle, 4.8% user, 5.3% kernel, 83.4% iowait, 0.0% swap
Memory: 8192M real, 6839M free, 600M swap in use, 9881M swap free

   PID USERNAME LWP PRI NICE SIZE RES STATE TIME CPU COMMAND
 10585 oracle 11 60 0 362M 328M sleep 132:05 3.84% oracle
 25195 oracle 1 59 0 360M 324M sleep 0:02 0.89% oracle
 25142 oracle 1 59 0 360M 326M sleep 0:01 0.61% oracle
 24928 oracle 1 60 0 360M 324M sleep 0:45 0.55% oracle
 19068 oracle 1 59 0 360M 326M sleep 11:16 0.24% oracle
   415 oracle 13 59 0 370M 314M sleep 486:28 0.19% oracle
 25166 oracle 1 59 0 360M 324M sleep 0:00 0.15% oracle
 25146 oracle 1 59 0 360M 324M sleep 0:01 0.12% oracle
 25228 root 1 59 0 2896K 1768K cpu/0 0:00 0.10% top
 25170 oracle 1 59 0 360M 324M sleep 0:02 0.08% oracle
 25497 oracle 1 59 0 2960K 1776K sleep 8:16 0.08% top
 25158 oracle 1 59 0 360M 326M sleep 0:01 0.06% oracle
 25178 oracle 1 59 0 360M 326M sleep 0:00 0.06% oracle
 25186 oracle 1 59 0 360M 324M sleep 0:00 0.05% oracle
 25154 oracle 1 59 0 360M 324M sleep 0:00 0.05% oracle

root@AD-Oracle-01# uname -a
SunOS AD-Oracle-01 5.9 Generic_118558-14 sun4u sparc SUNW,Sun-Fire-480R

root@AD-Oracle-01# iostat 5 15
   tty sd0 sd16 sd17 sd22 cpu
 tin tout kps tps serv kps tps serv kps tps serv kps tps serv us sy wt
id
   0 12 0 0 1 0 0 0 0 1 0 0 0 0 11 3 18
68
   0 182 0 0 0 0 0 0 0 1 0 0 0 0 4 3 72
21
   0 140 0 0 0 0 0 0 0 1 0 0 0 0 13 5 58
25
   0 136 0 0 0 0 0 0 0 1 0 0 0 0 6 2 80
12
   0 143 0 0 0 0 0 0 0 1 0 0 0 0 18 4 48
29
   0 136 0 0 0 0 0 0 0 1 0 0 0 0 8 5 67
21
   0 125 0 0 0 0 0 0 0 1 0 0 0 0 4 3 67
26
   0 99 0 0 0 0 0 0 0 1 0 0 0 0 4 4 63
30
   0 112 0 0 0 0 0 0 0 1 0 0 0 0 4 2 73
20
   0 124 0 0 0 0 0 0 0 1 0 0 0 0 15 2 31
52
   0 108 0 0 0 0 0 0 0 1 0 0 0 0 5 2 65
28
   0 109 0 0 0 0 0 0 0 1 0 0 0 0 7 6 65
22

root@AD-Oracle-01# iostat -xnp
                    extended device statistics
    r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.8 0 0 c0t0d0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd16
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd16,a
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd16,b
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd16,c
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd16,g
    0.7 0.1 0.1 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0
    0.7 0.1 0.1 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0s0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0s1
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0s2
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c2t1d0s6
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 sd22
    1.3 10.5 44.4 82.9 0.0 0.1 0.2 8.2 0 6 c1t0d0
    0.5 0.3 1.0 0.6 0.0 0.0 2.7 10.8 0 0 c1t0d0s0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 9.4 0 0 c1t0d0s1
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t0d0s2
    0.1 0.8 6.6 6.6 0.0 0.0 0.5 15.2 0 1 c1t0d0s4
    0.6 9.5 36.7 75.7 0.0 0.1 0.0 7.4 0 6 c1t0d0s5
    0.0 0.0 0.0 0.0 0.0 0.0 0.7 38.5 0 0 c1t0d0s7
   18.2 26.7 626.1 246.0 0.0 1.1 0.0 25.3 0 26 c1t1d0
    0.6 0.3 0.2 0.0 0.0 0.0 0.0 0.0 0 0 c1t1d0s0
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t1d0s1
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t1d0s2
   17.6 26.4 626.0 246.0 0.0 1.1 0.0 25.8 0 26 c1t1d0s6
    0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0 0
AD-Oracle-01:vold(pid323)
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers



This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:41:07 EDT