SUMMARY Fragmenting AdvFS and sync IO

From: Lucas Carey (lcarey@odd.bio.sunysb.edu)
Date: Fri Jul 26 2002 - 12:26:28 EDT


Thanks to Bob Harris and Alan,

Frag File:
Anyway, a file 'frag' is the last part of a file that does not use
up the entire 8K if the amount of wasted space in the last 8K of the
file is more than 5% of the file's total storage. If a file wastes
less than 5% of the total space allocated, it does _NOT_ get a frag.
Generally this means that any file greater than 163K will never get
a frag, but again depending on the percentage of waste, smaller
sized files may not have frags either.

Frag sizes range from 1K to 7K and are stored together in the AdvFS
"frag file".

Disabling frags means that AdvFS will not apply the 5% rule when
closing a file and will leave the tail end of the file where it is.
So yes, more space will be used for small files.

AdvFS also keeps track of whether a file has a frag and where that frag
is located in the "frag file".

In the presense of a frag file, defragment may try to
put all the small files into the frag file, but normal
file system operation would do the same.

Defragmenting Filesystems:
  A fragmented file is one where all of the storage for the file is
_NOT_ contiguous on disk. A fragmented file system is one where the
free space on the disk is _NOT_ contiguous. The defragment utility tries
to rearrange files on the disk to get contiguous files and as much
contiguous free space as it can. Getting everything perfectly
defragmented is never possible because there are a few file system
metadata files that can not be moved so these may break up free space or
a file may need to be located before and after a system metadata file
(or some part of a metadata file). However, that is the exception case.

     AdvFS always keeps track of a file's data, whether it is contiguous
     or fragmented on the disk. The information is kept in an extent map
     as metadata for each file.
     AdvFS never actually kept any "separate" information about what
     files were fragmented and what files were contiguous.

     Since there is no special information kept about fragmented or
     defragmented files, the defragment utility finds out the state of
     domain fragmentation by scanning _ALL_ the extent maps for all the
     files on the disk and figures out how it wants to move things around
     to defragment the domain.

And on synchronous writes for /var/spool/mail/

It depends on why the system crashes. There are lots of
crash causes that are also the cause of corruption. A
power failure may cause the disks to stop writing in the
middle of I/O. Memory or data path problems can cause
data corruption directly. One hopes that ECC and parity
protection notice it, but there is small chance it won't.

Still, if you have the I/O overhead to afford the potential
loss of performance by going to synchronous writes, it may
improve data reliability in some cases.

--Original Question--
How are the frag file and defragmenting the filesystem related? On a
system with lots of spare space, will there be a performance gain from
disabling the frag file?
Are there any thoughts on forcing ail spool files to do synchronous I/O
for a mail server that does very little disk I/O?



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:48:47 EDT