Re: root filesystem shows 100% full when it isn't

From: Ray Schafer (schafer@TKG.COM)
Date: Sun Dec 15 2002 - 15:54:16 EST


I have had that exact thing happen a few times. I didn't see the
resolution of the thread, but here is something that can be done to take
a peek at, and remove what might be under the mount points:

If you nfs export "/" as read-write root access to localhost, and then
nfs mount it under /mnt (mount localhost:/ /mnt), you can cd to /mnt and
then quickly tell if there are files under the mount point. When you
mount a filesystem, the mount points under that are not mounted,
allowing you to see what's under them. Having it mounted with
read-write access by root allows you to modify (or remove) the files
under the mountpoints without having to go into maintenance mode.

Aaron Morris wrote:

> I believe the reason John's / filesystem was full (if I remember the
> posts correctly) was files in a directory that had been mounted over so
> he could not see the files without umounting the filesystem.
>
> pSeries AIX Geek wrote:
>
>> I know that this is an old post, but I just had to
>> comment.
>>
>> The reason that his / is 100% full with no seemingly
>> large files is because there is a process running
>> holding open a file that's been deleted. Using fuser
>> (try the -dV or -cx options) will probably find it.
>>
>> Usually, there isn't THAT much data in /, so the
>> benefits of making the nbpi smaller are really
>> negligible (plus, you'd EITHER have to hack your
>> image.data file ahead of time or use Sysback to
>> restore; there's no way to do a FRESH install and
>> modify the nbpi on / ).
>>
>> - pAG
>> --- Aaron Morris <aaronmorris@MINDSPRING.COM> wrote:
>>
>>> This reminds me of a problem I had on Windows
>>> systems using FAT16. A
>>> 2GB FAT16 filesystem has a 32KB cluster (usually
>>> called inodes in unix)
>>> size. A 1KB file wastes space on a filesystem
>>> because it makes the
>>> other 31KB of the cluster it occupies unusable. In
>>> one circumstance I
>>> saw 1.2GB of files actually taking up 2GB of space
>>> because of cluster
>>> size. By converting the filesystem to NTFS (with
>>> 4KB cluster size), it
>>> essentially "freed" ~600MB of space. This is more
>>> of an attempt at an
>>> explanation than a solution.
>>>
>>> You could try and recreate the "/" filesystem with a
>>> smaller inode size.
>>> The default inode size for JFS is 4KB.
>>>
>>> Barry Deevey wrote:
>>>
>>>> Hello again,
>>>>
>>>> We've now managed to boot from mksysb and run fsck
>>>
>>>
>>> on the root filesystem,
>>>
>>>> but fsck did not display any problems, but / is
>>>
>>>
>>> still 100% full - I have
>>>
>>>> thoroughly checked the system using ls -al and du
>>>
>>>
>>> -s * on root and cannot
>>>
>>>> see anything at all that could possibly use up all
>>>
>>>
>>> of the space (as / has
>>>
>>>> over 2Gb allocated to it)
>>>>
>>>> Is there anything else we can try before we are
>>>
>>>
>>> forced to restore from
>>>
>>>> mksysb???
>>>>
>>>> Cheers,
>>>>
>>>> Barry.
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Barry Deevey
>>>
>>>
>>> [mailto:Barry.Deevey@SEFAS.CO.UK]
>>>
>>>> Sent: Monday, December 02, 2002 7:57 AM
>>>> To: aix-l@Princeton.EDU
>>>> Subject: Re: root filesystem shows 100% full when
>>>
>>>
>>> it isn't
>>>
>>>>
>>>> Can't seem to find the steps to get to that menu
>>>
>>>
>>> option having booted from
>>>
>>>> CD. Can you help?
>>>>
>>>> Barry
>>>>
>>>> -----Original Message-----
>>>> From: Bill Verzal [mailto:Bill_Verzal@BCBSIL.COM]
>>>> Sent: 02 December 2002 15:36
>>>> To: aix-l@Princeton.EDU
>>>> Subject: Re: root filesystem shows 100% full when
>>>
>>>
>>> it isn't
>>>
>>>>
>>>> "Access a volume group before mounting
>>>
>>>
>>> filesystems" from the maintenance
>>>
>>>> menu.
>>>>
>>>
>> ----------------------------------------------------------------------------
>>
>>
>>>> ----------------------------
>>>>
>>>> Bill Verzal
>>>> Technical Consultant
>>>> Forbes Technical Consulting
>>>> (312) 653-3684
>>>> bill_verzal@bcbsil.com
>>>> billverzal@imcingular.com (Pager)
>>>> 888-428-4025 (Pager)
>>>> MailStop: 27.202B
>>>>
>>>>
>>>>
>>>> "Barry Deevey"
>>>> <Barry.Deevey@SE To:
>>>
>>>
>>> aix-l@Princeton.EDU
>>>
>>>> FAS.CO.UK> cc:
>>>> Sent by: "IBM
>>>
>>>
>>> Subject: Re: root filesystem
>>>
>>>> shows 100% full when it isn't
>>>> AIX Discussion
>>>> List"
>>>> <aix-l@Princeton
>>>> .EDU>
>>>>
>>>>
>>>> 12/02/2002 09:40
>>>> AM
>>>> Please respond
>>>> to "IBM AIX
>>>> Discussion List"
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I have tried booting into single user mode and
>>>
>>>
>>> running fsck /dev/hd4 (root
>>>
>>>> filesystem) but this will not try to fix anything
>>>
>>>
>>> as the filesystem is
>>>
>>>> still
>>>> mounted. Can anybody help in explaining how I can
>>>
>>>
>>> run fsck on a root??
>>>
>>>> Thanks for any replies, they're much appreciated.
>>>>
>>>> -----Original Message-----
>>>> Sent: 02 December 2002 13:14
>>>> To: aix-l@Princeton.EDU
>>>> Subject: root filesystem shows 100% full when it
>>>
>>>
>>> isn't
>>>
>>>>
>>>> df shows / filesystem as 100% full, but ls and df
>>>
>>>
>>> do not show sufficient
>>>
>>>> data to fill it.
>>>>
>>>> Tried a reboot but to no effect.
>>>>
>>>> Any ideas?
>>>>
>>>> John Dunn
>>>> Sefas Innovation Ltd
>>>> 0117 9154267
>>>> www.sefas.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> **********
>>>> The information contained in this communication is
>>>
>>>
>>> confidential, private,
>>>
>>>> proprietary, or otherwise privileged and is
>>>
>>>
>>> intended only for the use of
>>>
>>>> the
>>>> addressee. Unauthorized use, disclosure,
>>>
>>>
>>> distribution or copying is
>>>
>>>> strictly prohibited and may be unlawful. If you
>>>
>>>
>>> have received this
>>>
>>>> communication in error, please notify the sender
>>>
>>>
>>> immediately at
>>>
>>>> (312)653-6000 in Illinois; (972)766-6900 in Texas;
>>>
>>>
>>> or (800)835-8699 in New
>>>
>>>> Mexico.
>>>> **********
>>>>
>>>
>>> --
>>> Aaron W Morris
>>
>>
>>
>>
>> __________________________________________________
>> Do you Yahoo!?
>> Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
>> http://mailplus.yahoo.com
>>
>
> --
> Aaron W Morris
>

--
Ray Schafer - WFSO NetBackup/Bare Metal Restore Team: (512) 433-3345, cell (512) 426-3188
My complete contact info is at  http://wfso.veritas.com/bio/schafer_ray
Got questions?  The VIC http://wfso.veritas.com/vic is how WFSO answers them!


This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:16:24 EDT