SUMMARY: Temporarily vanishing diskspace

From: Bütow, Michael (michael.buetow@comsoft.de)
Date: Wed Oct 08 2003 - 06:44:20 EDT


Many thanks to Selden E. Ball, Dr Thomas Blinn, Jim Belonis
and Alan Rollow for making things clear:

The "vanishing space" was most likely consumed by a program
creating a large temporary file (on my system /tmp is a link
to /usr/users/tmp , so that would fit the symptoms).

Using the lsof(8) program I would have been able to find out
the culprit process if I had not gone to single user mode,
thereby killing the guilty process and closing its file(s).

Now I have to wait until the problem reappears to find out
which process causes it.

I append the lucid explanation given by Alan:

        du(1) can only count the space used by files that have
        names. A common technique for creating a temporary file
        is to open the file, remove the name, and then use it
        normally. When the file is closed, it will be deleted
        by the kernel, so the creating program doesn't need to
        worry about it.

        Your missing space has all the fingerprints of a large
        temporary file. Such files tend to end up in /tmp or
        /var/tmp, but a user knowing the file is going to be
        big, could have set TMPDIR to be space on his home
        directory (which he might expect to have more, if not
        enough in this case, space).

        When you shut down to single user, the process stopped
        and the file was deleted, freeing the space.

        Track down a copy of lsof, and have that available if
        it happens again. It should be able to show open files,
        and the processes that own them. Hopefully, it will
        show files that don't have names and their sizes, which
        will offer a good clue.

[Note: lsof(8) is not installed per default with Tru64 Unix, but I
think it is delivered with the Freeware or Software Products
Library CD-ROMs. ]

Regards,
Michael

---------------------------------

The original question was:

Hi managers,

I have freshly installed a Tru64 4.0F box, with patchkit 7.

It was left running overnight to find the next morning that the
/usr/users filesystem (a 5GB UFS partition) was full.
Indeed 'df -k' showed capacity at 101% .
The messages log contained a disk full message and a lot of
"message repeated" messages with huge repetition counts.

I logged in and did a 'du -sk' on /usr/users, and it showed
me that approx 1.6GB was used - around 33 %).

Into single user, and forced a fsck of the partition. All clean.
Rebooted to multiuser - now 'df -k' shows a more "correct" figure of about
36% usage.

The machine has been running since and the problem has not recurred, but I have never experienced this before.

I've seen others in this group have problems, but they all seem to have been on AdvFS.

Any ideas what could cause this?

Kind regards,
Michael Bütow



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:49:38 EDT