From: Rebstock, Roland (Roland.Rebstock@usi.net)
Date: Mon Oct 02 2006 - 10:08:44 EDT
Sun Managers, I have SUN Fire V880 with Solaris 8 and veritas volume
manger 3.5 running and the servers IO Wait through top is 85%. Looking
a iostat and qiostat there is very little disk activity. It seems to
bound in ram as vmstat is reporting upto 20 processes in the blocked
queue. Any suggestions? This does not seems to by physical I/O but
rather memory/cpu IO bound.
CPU states: 3.0% idle, 5.8% user, 2.5% kernel, 88.7% iowait, 0.0%
swap
^Cmsuapd1g:> ./qiostat -i 1 -c 10 /orafiles
Checking memory statistics...
procs memory page disk faults cpu
r b w swap free re mf pi po fr de sr s6 sd sd sd in sy
cs us sy id
0 1 0 7967384 3951944 434 1268 1832 0 0 0 0 0 23 22 4 735 15731
5473 11 3 86
0 19 0 7666016 3752608 185 1546 0 0 0 0 0 0 30 74 1 1177
6141 2889 14 3 83
0 18 0 7679800 3762480 91 30 0 0 0 0 0 0 69 111
4 1330 1590 2722 4 2 95
0 16 0 7691720 3773872 0 0 0 0 0 0 0 0 45 55
0 1014 1143 2016 2 1 96
0 16 0 7691720 3773872 0 0 0 0 0 0 0 0 57 7
1 1171 2951 2376 3 1 96
OPERATIONS FILE BLOCKS AVG TIME(ms)
FILE NAME READ WRITE READ WRITE READ
WRITE
Mon Oct 02 11:00:13 2006
/orafiles 0 0 0 0 0.0 0.0
Mon Oct 02 11:00:14 2006
/orafiles 0 0 0 0 0.0 0.0
Mon Oct 02 11:00:15 2006
/orafiles 0 0 0 0 0.0 0.0
Mon Oct 02 11:00:16 2006
/orafiles 0 0 0 0 0.0 0.0
Mon Oct 02 11:00:17 2006
/orafiles 0 0 0 0 0.0 0.0
-----Original Message-----
From: Super-User [mailto:root@msuapd1g.usi.net]
Sent: Monday, October 02, 2006 9:58 AM
Subject:
msuapd1g system diagnostics on Mon Oct 2 10:57:17 ADT 2006
Checking swap space...
total: 2748280k bytes allocated 7695040k available
Checking processes in WAIT state...
There are 4 waiting to clear...
tty cpu
tin tout us sy wt id
0 0 2 1 90 7
extended device statistics
r/s w/s Mr/s Mw/s wait actv wsvc_t asvc_t %w %b device
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c0t6d0
6.9 1.0 0.1 0.0 0.0 0.0 0.0 6.0 0 5 c1t3d0
34.7 1.0 0.3 0.0 0.0 0.1 0.0 1.6 0 6 c1t4d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t2d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t1d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c1t13d0
113.0 1.0 0.9 0.0 0.0 4.5 0.0 39.7 0 100 c1t9d0
97.2 2.0 0.8 0.0 0.0 4.0 0.0 40.3 0 100 c1t11d0
115.0 1.0 0.9 0.0 0.0 0.7 0.0 6.1 0 52 c1t10d0
15.9 4.0 0.1 0.0 0.0 0.3 0.0 14.4 0 28 c1t12d0
21.8 2.0 0.2 0.0 0.0 0.1 0.0 4.9 0 12 c6t3d0
30.7 2.0 0.2 0.0 0.0 2.4 0.0 73.2 0 84 c6t9d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t13d0
0.0 1.0 0.0 0.0 0.0 0.0 0.0 8.3 0 1 c6t0d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0 c6t1d0
2.0 2.0 0.0 0.0 0.0 0.0 0.0 9.6 0 4 c6t4d0
25.8 0.0 0.2 0.0 0.0 1.0 0.0 39.9 0 77 c6t11d0
27.8 2.0 0.2 0.0 0.0 0.4 0.0 12.5 0 35 c6t10d0
0.0 1.0 0.0 0.0 0.0 0.0 0.0 9.9 0 1 c6t2d0
45.6 2.0 0.4 0.0 0.0 1.1 0.0 23.7 0 46 c6t12d0
19.8 2.0 0.2 0.0 0.0 0.2 0.0 9.1 0 17 c6t8d0
23.8 4.0 0.2 0.0 0.0 0.1 0.0 5.3 0 14 c1t8d0
38.7 0.0 0.3 0.0 0.0 0.1 0.0 3.1 0 11 c6t5d0
13.9 2.0 0.1 0.0 0.0 0.0 0.0 3.1 0 5 c1t5d0
0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0 0
msuapd1g:vold(pid536)
tty cpu
tin tout us sy wt id
0 0 3 2 93 2
Checking Processor Stats...
CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt
idl
0 153 0 1088 7 1 703 3 11 31 0 2016 11 3 11
75
1 160 0 1121 9 3 696 3 10 31 0 1971 11 3 11
75
2 153 0 1124 7 1 684 3 11 30 0 1927 11 3 11
75
3 157 0 2002 192 186 611 3 10 29 0 1838 10 3 7
80
4 159 0 2094 200 194 613 3 10 29 0 1879 10 4 7
80
5 159 0 1104 160 153 660 3 10 39 0 1920 11 3 10
76
6 165 0 1186 7 1 728 3 11 32 0 2006 11 3 11
74
7 159 0 1227 249 144 774 3 10 33 0 2172 12 3 11
74
CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt
idl
0 0 0 85 4 1 239 0 6 7 0 161 2 0 98
0
1 0 0 1 8 5 243 0 7 1 0 103 2 0 98
0
2 0 0 2 4 1 272 0 5 16 0 130 5 1 67
27
3 0 0 1 362 359 214 0 8 6 0 101 2 2 96
0
4 0 0 2 405 402 249 0 5 18 0 127 2 1 97
0
5 0 0 1 6 1 443 1 8 11 0 226 5 0 69
26
6 0 0 1 7 1 218 2 8 9 0 337 19 1 77
3
7 5 0 4 350 247 198 0 13 14 0 92 2 1 97
0
Checking average wait for disk and network I/O...
Average number of jobs in the run queue over the last 1, 5 and 15
minutes
is: load average: 0.68, 1.00, 0.90
last pid: 16677; load averages: 0.67, 0.99, 0.90 10:57:34
144 processes: 140 sleeping, 2 zombie, 2 on cpu
CPU states: 11.3% idle, 6.3% user, 2.6% kernel, 79.8% iowait,
0.0% swap
Memory: 8192M real, 3715M free, 2785M swap in use, 7544M swap free
PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU
COMMAND
14298 root 1 11 0 7304K 5440K sleep 0:09 0.63% smbd
192 oracle 1 58 0 2051M 2021M sleep 5:00 0.29%
oracle
28160 oracle 1 58 0 2042M 2012M sleep 0:49 0.27%
oracle
18359 oracle 1 58 0 2044M 2015M sleep 3:18 0.25%
oracle
9622 oracle 1 55 0 2051M 2022M sleep 0:56 0.23%
oracle
17491 oracle 1 28 0 2053M 2024M sleep 1:16 0.17%
oracle
16858 oracle 1 58 0 2042M 2013M sleep 3:30 0.17%
oracle
10728 oracle 1 33 0 2047M 2017M sleep 0:02 0.16%
oracle
9578 oracle 1 28 0 2050M 2021M sleep 0:55 0.15%
oracle
13869 oracle 1 58 0 2042M 2012M sleep 1:19 0.14%
oracle
19402 oracle 1 58 0 2042M 2012M sleep 3:25 0.13%
oracle
4351 oracle 1 58 0 2042M 2012M sleep 0:38 0.13%
oracle
12174 oracle 1 52 0 2046M 2016M sleep 0:02 0.12%
oracle
7955 oracle 1 36 0 2047M 2018M sleep 20:57 0.11%
oracle
16478 oracle 1 58 0 2038M 2009M sleep 0:01 0.11%
oracle
_______________________________________________
sunmanagers mailing list
sunmanagers@sunmanagers.org
http://www.sunmanagers.org/mailman/listinfo/sunmanagers
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 23:40:55 EDT