From: John F Riordan (jriorda2@CSC.COM)
Date: Tue Aug 05 2003 - 15:39:45 EDT
Hi all,
I have a 7026-6H1 4 processors 6GB Ram
Storage from EMC 36, 8GB luns.
AIX 5.1 ML-01
Oracle 8.1.7
Sybase 12.5
Today the machine locked up, for the first time in the three years we have
had it. I booted system as I had a clean system dump. As the system came
up my Oracle file system did not mount. As that file system was corrupted.
I ran fsck and the file system was fine. Once Oracle and Sybase started, I
worked with IBM as they analyzed the dump file. I was told the system
crashed due to a corrupted file system and that file system was our Oracle
application directory.
I am trying to find what could have caused the corrupted file system.
My errpt goes back to Sep 2002 and there are no errors for "Hardware" "Full
file systems" "jfs_log file size".
I started looking at the Oracle instance logs "adhoc, bdump" etc.. The
only thing I noticed was that one alert log for an instance was smaller
than the others. When I looked at it the start of the log was when the
system came back up. The only data in that log was from today after the
crash. All the other alert logs go back to the beginning of July. I
noticed one of the Oracle DBA's was logged in when the system crashed. At
the time of the crash he was purging the alert log file that is now
smaller. Thought he might have deleted the file instead of purge. The
instance of Oracle was still up and running at the time he was in this
file.
I noticed in the "kdb.out" file generated by IBM there is a list of each
CPU with PSLOT PID etc.. On one of the CPU's there under the PROC_NAME
there is "vi"
(1) > status ^M
CPU TID TSLOT PID PSLOT PROC_NAME^M
0 205 2 204 2 wait^M
1 1ABB9 427 7044 112 vi^M
2 409 4 408 4 wait^M
3 50B 5 50A 5 wait^M
Again, I know this is a long shot, but was wondering if anyone had a
thought as to what might have happened.
Thanks for taking the time to read this.
John
John Riordan
Unix Systems Administrator
CSC / Bath Iron Works
Bath, Maine
jriorda2@csc.com
207.442.1094
----------------------------------------------------------------------------------------
This is a PRIVATE message. If you are not the intended recipient, please
delete without copying and kindly advise us by e-mail of the mistake in
delivery. NOTE: Regardless of content, this e-mail shall not operate to
bind CSC to any order or other contract unless pursuant to explicit written
agreement or government initiative expressly permitting the use of e-mail
for such purpose.
----------------------------------------------------------------------------------------
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:17:07 EDT