From: Theresa Sarver (IFMC.tsarver@SDPS.ORG)
Date: Fri Jul 26 2002 - 16:38:39 EDT
Hello;
Environment:
SP Complex (9076) 1 frame, 7 nodes
AIX 4.3.3 ML 8
PSSP 3.2
PTFSET 8
I've been on leave for the past few weeks and have just returned to
find that our SP Admin has left the firm and I'm now in charge of the SP
complex. More importantly, I have "very limited" experience with the
SP, so I'm be relying on all you SP experts quite a lot until I'm up to speed.
The issue I'm currently having is NOT SP related (well, I don't think anyway). On July 11, around 11AM the (now gone) SP Admin updated the /etc/inetd.conf file and commeted out the following on NODE5:
exec, ntalk, rusersd, sprayd, pcnfsd, time, dtspc, cmsd, ssalld
About an hour later (so I'm told anyway) users started calling and saying that they couldn't get into NODE5. All users were getting the following error message:
3004-009: Failed Running Login Shell
The SP Admin was able to log in as user root fom the CWS. At which point she called IBM for assistance. IBM immeidately noticed that the file permissions on /usr, /usr/bin, /etc were all 700 - they changed them to 755...still no one could log in. So the SP Admin restored from a mksysb image and all appeared to be fine.
After ensuring people could log in, the Admin rebooted one final time to ensure everything was 'okay'. however, when the server came back up all the permissions were screwed up again and once again no one (other than root) could log in. She restored from mksysb again, and then left the firm shortly thereafter - without resolving the issue. This node has not been rebooted since, and I'm at a loss as to where to look to try to fix this issue.
IBM is saying that an application, with root privilages, which is starting at boot-time is changing these permissions. The problem is that this is an SP Complex and almost everything in NODE5's /etc/inittab is starting on several other nodes as well - and they aren't experiencing problems? The only difference that I can see between the nodes is that NODE5 has the following line, why the other nodes do not:
orapw:2:wait:/etc/loadext -l /etc/pw-syscall 2>&1
NODE5 currently is not running Oracle, if that's relevant. - Anyone know what this line does? Is it safe to comment it out?
Otherwise, all the remaining nodes are also loadind the following software applications.
adsm:2:respawn:/usr/bin/dsmc sched > /dev/null 2>&1 # TSM scheduler
connect:2:respawn:/usr/local/connect/start.cdpmgr /tmp/connect.log 2>&1
orapw:2:wait:/etc/loadext -l /etc/pw-syscall 2>&1
orakstat:2:wait:/etc/loadext -l /etc/ora_kstat 2>&1
:oracle:2:wait:/c2f1n5in/u01/oracle/product/8.1.7/bin/start_oracle > /tmp/oralog
express:2:wait:/home/oracle/expstart > /tmp/wwwexp.log 2>&1
apache:2:wait:/scripts/apachestart.sh > /tmp/apache.log 2>&1
imnss:2:once:/usr/IMNSearch/bin/imnss -start imnhelp >/dev/console 2>&1
imqss:2:once:/usr/IMNSearch/bin/imq_start >/dev/console 2>&1
Has anyone seen this before? Or does anyone have any ideas on where I can start? If this is an application issue - why would this start almost entirely "out of the blue"?
Thanks in advance for the help - and I apologize for such a long post.
Theresa
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:16:05 EDT