From: David.Knight@clubcorp.com
Date: Wed Mar 10 2004 - 09:21:11 EST
Managers
Thanks to Steve VanDevender, Oisin McGuinness, Bob Harris, Dr. Alan
Rollow, James Fitzgerald, Bob Vickers and Bård Tesaker for there help and
time on explaining the behavior of my file domain. It appears that there
isn't a problem at all.
The behavior that I seeing from the /data5 filesystem is due to the way
that our version of oracle allocates space in it's temp files.
"Thomas, Thomas" provided this nugget of knowledge from the Oracle
webpage:
___________________________________________________________________
fact: Oracle Server - Enterprise Edition 8.1
fact: tempfile
symptom: space for a tempfile not allocated in the filesystem
symptom: space not completely used in filesystem
cause: A temporary tablespace with a tempfile is added to the database:
create temporary tablespace test_temp TEMPFILE '/oracle/test_temp.dbf'
size 400m reuse extent management local uniform size 1024k;
You see the correct filesize with a 'ls -l', but a df -k reports too much
free
space in that filesystem.
Tempfiles are not fully allocated and initialized during creation or
resize.
The description for this performance feature was omitted in Oracle8i.
fix:
The Oracle9i Database Concepts Release 1 guide has been updated to reflect
this
behaviour.
For example:
Locally managed temporary tablespaces have temporary datafiles (tempfiles)
which are similiar to ordinary datafiles except that:
When you create or resize tempfiles, they are not always guaranteed
allocation
of disk space for the filesize specified. On certain file systems (for
example
UNIX) disk blocks are allocated not at file creation or resizing, but
before the
blocks are accessed.
Caution: This enables fast tempfile creation and resizing;
however the disk could run out of space later when the tempfiles are
accessed.
___________________________________________________________
In essence our oracle temp files are "sparse" files as mentioned by
several people above. Sparse files are files that have a "hole" in them (a
range of empty bytes). So for example:
# ls -ls TEMP01.dbf
197280 -rw-r----- 1 oracle system 209723392 Mar 10 06:51 TEMP01.dbf
# du -kx ./*
197280 ./TEMP01.dbf
80 ./TEMP03.dbf
1 ./test.script
13896 ./test.test
# ls -l
total 211257
-rw-r----- 1 oracle system 209723392 Mar 10 06:51 TEMP01.dbf
-rw-r----- 1 oracle system 209723392 Mar 5 10:46 TEMP03.dbf
-rw-r--r-- 1 root system 69 Feb 23 08:09 test.script
-rw-r--r-- 1 root system 14226971 Mar 9 10:19 test.test
# pwd
/data5/u002/oradata/prod
The TEMP01.dbf file has 209723392 bytes allocated but only has 197280
bytes used. A few weeks ago when I was seeing my file domain grow but
could not see any files growing, well the df, vdf, and showfdmn commands
show the file system space that is used. the ls command was displaying the
space that we had allocated of which was not changing. The space was being
used by oracle filling in the holes in the temp files (extents).
Thanks again for all of the quick responses and the time that I was given
from the people above. This case (for you HP folk: 3204932768) has been
worked by HP for about 3 weeks now and around six Technicians / Engineers
have tried to find the answer to my problem.
More info on sparse files can be found at:
www.cs.wisc.edu/~thain/library/sparse.pdf
-David Knight
----- Forwarded by David Knight/CLUBCORP/US on 03/10/2004 07:59 AM -----
David Knight
03/09/2004 01:01 PM
To: tru64-unix-managers@ornl.gov
cc:
Subject: Filesystem --- "ls -l" total reporting incorrect file sizes - ADVFS
Hello Managers,
Got an issue that HP has been trying to figure out for some time
now. Thought maybe I'd give you guys a shot at it.
Problem=
Filesystem has issue with files not showing the correct size.
there are only a few oracle .dbf files on the file system and when I
preform an ls -l on the directory:
# pwd
/data5/u002/oradata/prod
# ls -l
total 205553
-rw-r----- 1 oracle system 209723392 Mar 9 12:46 TEMP01.dbf
-rw-r----- 1 oracle system 209723392 Mar 5 10:46 TEMP03.dbf
-rw-r--r-- 1 root system 69 Feb 23 08:09 test.script
-rw-r--r-- 1 root system 14226971 Mar 9 10:19 test.test
the file sizes added up don't equal the "total" that ls is reporting for
the total size of the files in the current directory. a few weeks ago when
the problem was found I noticed that there was a .dbf file that was
growing in size yet the ls -l size for the file was not reporting any
change. the date was being updated for the file but the size was not
reporting any change. now don't get this confused with the df issue
showing the quota info... df, showfdmn, and vfd all show the correct info
for the file system/file domain. HP first had be shutdown my entire
cluster. just by chance we had a power outage at the same time as I was
shutting down the cluster and our UPS ran out of juice so the entire
cluster/memory channels/san was powerless to the point of the write back
cache batteries being drained. this showed no change in the file system
issue so HP had me run verify witch showed not issues then a fixfdmn witch
showed some messages:
21:18:26 | FIXED | LOG | /dev/disk/dsk83c:56880000 | Modified log page 0
so the log is marked as empty.
21:18:26 | FIXED | LOG | /dev/disk/dsk83c:56880016 | Modified log page 1
so the log is marked as empty.
21:18:26 | FIXED | LOG | /dev/disk/dsk83c:56880032 | Modified log page 2
so the log is marked as empty.
21:18:26 | FIXED | LOG | /dev/disk/dsk83c:56880048 | Modified log page 3
so the log is marked as empty.
21:18:26 | FIXED | LOG | /dev/disk/dsk83c:56880064 | Modified log page 4
so the log is marked as empty.
21:18:26 | FIXED | LOG | /dev/disk/dsk83c:56880080 | Modified log page 5
so the log is marked as empty.
this message repeats it's self in the fixfdmn log file up to page 511. So
HP then had me remove and recreate the file domain. after that fixfdmn
reports the same message. after talking with 4 people from HP I was told
not to worry about this message if it appears after the file domain was
recreated. Well a week later (Now) I have the problem still.. Not sure
what else to do on this one. any help would be grateful..
O,the version info is:
Tru64 5.1 / TruCluster 5.1 - ADVFS file system
Patch:
- T64V51AS0003-20010521 OSF510
- T64V51AS0003-20010521 TCR510
- T64V51B19AS0005-20020411 OSF510
- T64V51B19AS0005-20020411 TCR510
Thanks in Advance,
David Knight
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:49:53 EDT