99% CPU Usage

From: Bona Craig (bona_64unix@yahoo.com)
Date: Wed Sep 03 2003 - 11:31:46 EDT


Hi all,
I have a 3-server ES40 cluster. for a while now, the
server has been recording a 99% cpu utilization. (see
beneath)
______________________________________________________load
averages: 1.89, 1.85, 1.88
                                                      
                                           13:49:10
131 processes: 4 running, 39 waiting, 33 sleeping, 54
idle, 1 zombie
CPU states: 1.9% user, 21.1% nice, 27.0% system,
49.7% idle
Memory: Real: 690M/4000M act/tot Virtual: 3M/8576M
use/tot Free: 2848M

  PID USERNAME PRI NICE SIZE RES STATE TIME
CPU COMMAND
1954000 root 51 0 2544K 335K run 17.7H
99.90% ksh
2041879 sas 44 8 33M 11M run 51:01
94.30% sas
1572864 root 0 0 14G 177M run 367:23
1.50% kernel idle
2042313 root 48 4 9296K 6144K run 0:00
0.30% top
2039363 sas 44 4 50M 12M sleep 45:05
0.00% sas
2041331 sas 44 4 55M 22M sleep 23:13
0.00% sas
2039297 sas 44 4 52M 13M sleep 18:31
0.00% sas
1573016 root 44 0 2064K 114K sleep 12:12
0.00% update
1573868 root 43 -1 8584K 4169K sleep 5:16
0.00% advfsd
1573953 root 44 0 6304K 843K sleep 3:27
0.00% bprd
1700458 root 44 0 10M 3678K sleep 1:11
0.00% Xdec
1574352 dpm 44 0 9360K 3629K sleep 0:29
0.00% java
1573607 root 44 0 4784K 704K sleep 0:17
0.00% gated
1574552 root 44 0 33M 25M sleep 0:13
0.00% smsd
1573798 root 44 0 3280K 589K sleep 0:10
0.00% clu_mibs

okan.mtn.com.ng:/> ps -ef |grep 1954000
root 1954000 1954001 99.9 17:32:57 pts/4
17:41:10 -ksh (ksh)
root 2042240 2040686 0.0 13:50:09 pts/9
0:00.00 grep 1954000
okan.mtn.com.ng:/> ps -ef |grep 1954001
root 1954001 1573882 0.0 17:32:57 ??
0:00.02 telnetd
root 1954000 1954001 99.7 17:32:57 pts/4
17:42:51 -ksh (ksh)
root 2042377 2040686 0.0 13:51:51 pts/9
0:00.00 grep 1954001
okan.mtn.com.ng:/> ps -ef |grep 1573882
root 1573882 1573881 0.0 Aug 24 ??
0:13.77 -child (inetd)
root 1700509 1573882 0.0 Aug 25 ??
0:00.12 rpc.ttdbserverd
root 1954001 1573882 0.0 17:32:57 ??
0:00.02 telnetd
sas 2039211 1573882 0.0 08:32:14 ??
0:00.25 -uba1 (ftpd)
sas 2039247 1573882 0.0 08:36:07 ??
0:00.25 -uba1 (ftpd)
sas 2042066 1573882 0.0 13:21:45 ??
0:00.07 -uba1 (ftpd)
sas 2042149 1573882 0.0 13:39:14 ??
0:00.09 -uba1 (ftpd)
root 2042383 2040686 0.0 13:52:02 pts/9
0:00.00 grep 1573882
okan.mtn.com.ng:/> ps -ef |grep 1573881
root 1573881 1572865 0.0 Aug 24 ??
0:00.00 /usr/sbin/inetd
root 1573882 1573881 0.0 Aug 24 ??
0:13.77 -child (inetd)
root 2042392 2040686 0.0 13:52:15 pts/9
0:00.00 grep 1573881
okan.mtn.com.ng:/> ps -ef |grep 1572865 |more
root 1572865 1572864 0.0 Aug 24 ??
0:17.96 /sbin/init -a
root 1572933 1572865 0.0 Aug 24 ??
0:00.31 /sbin/kloadsrv
root 1572935 1572865 0.0 Aug 24 ??
0:00.07 /sbin/hotswapd
root 1573005 1572865 0.0 Aug 24 ??
0:00.03 /usr/sbin/esmd
root 1573016 1572865 0.0 Aug 24 ??
12:13.11 /sbin/update
root 1573132 1572865 0.0 Aug 24 ??
0:52.36 /usr/sbin/evmd
root 1573347 1572865 0.0 Aug 24 ??
0:00.67 /usr/sbin/niffd
________________________________________________

If I kill the Ksh process, the utilization drops. I
wonder if there is any way I can find the cause of the
problem instead of just killing the process.

Thanks

__________________________________
Do you Yahoo!?
Yahoo! SiteBuilder - Free, easy-to-use web site design software
http://sitebuilder.yahoo.com



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:49:34 EDT