jobs hanging

From: Bugs (bb1@humboldt.edu)
Date: Wed Apr 26 2006 - 12:20:51 EDT


ES40 Tru64 5.1B PK4.

I have a strange problem. Processes started hanging on our oracle
database server about a week ago. "dirclean" process started from
cron are all hung. Some "cp" commands copying files to one
filesystem are also hung.

Below are results from a dirclean that I started. The first time,
I started it without the trailing "/", and just got the error message.
The second time, I added the trailing "/", just like it comes from crontab.
It is now hung.

willow> /usr/sbin/dirclean -t +2 -o -n -k s /tmp
dirclean: Starting directory argument is not a directory: /tmp

(this one is hung)
willow> /usr/sbin/dirclean -t +2 -o -n -k s /tmp/

Here is the /tmp entry:
Filesystem 1024-blocks Used Available Capacity Mounted on
/dev/disk/dsk5a 257560 124538 120144 51% /
/dev/disk/dsk18a 548159 1696 519055 1% /cluster/members/member0/tmp

in fstab:
/dev/disk/dsk18a /tmp ufs rw 0 2

ls -l from root:
lrwxrwxrwx 1 root system 26 Mar 21 2001 tmp@ -> cluster/members/{memb}/tmp

I dont think that it is especially to do with /tmp.

Here are a few jobs hung up operating on /dsk14f2

root 125533 105795 0.0 Apr 20 pts/2 0:00.01 file TEMP.dbf USERS.dbf control01.ctl control02.ctl control03.ctl df1.dbf o1_mf_1_0h8pf400_.log o1_mf_1_0h8pv2v1_.log o1_mf_2_0h8pf9l8_.log o1_mf_2_0h8pv982_.log o1_mf_sys_undo_0h8pg83b_.dbf o1_mf_sys_undo_0h8pwby0_.dbf o1_mf_system_0h8pfm8t_.dbf o1_mf_system_0h8pvn74_.dbf o1_mf_temp_ts_0gvq3v4t_.tmp o1_mf_undo_ts_0gvq1tr7_.dbf redo_log11.log redo_log12.log redo_log21.log redo_log22.log
oracle 405940 1 0.0 Apr 19 ?? 0:06.96 cp control01.ctl control02.ctl control03.ctl o1_mf_1_0h8pf400_.log o1_mf_1_0h8pv2v1_.log o1_mf_2_0h8pf9l8_.log o1_mf_2_0h8pv982_.log redo_log11.log redo_log12.log redo_log21.log redo_log22.log TEMP.dbf USERS.dbf df1.dbf o1_mf_sys_undo_0h8pg83b_.dbf o1_mf_sys_undo_0h8pwby0_.dbf o1_mf_system_0h8pfm8t_.dbf o1_mf_system_0h8pvn74_.dbf o1_mf_undo_ts_0gvq1tr7_.dbf /dsk14f2/oradata/heattest/

(disk14f2)
/dev/disk/dsk14h 70448472 8078923 58847125 13% /dsk14f2

We rebooted last week, but the problem did not resolve.
So now cron has reached MAX jobs.

Does anyone know how to help me with this?
Thanks

Bugs

Operating Systems Analyst for unix systems
Humboldt State Univ. Information Technology Services
Arcata, Calif.

email bb1@humboldt.edu



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:50:29 EDT