From: rob.leadbeater@lynx.co.uk
Date: Mon Oct 07 2002 - 04:22:50 EDT
Hi Managers,
A *long* time ago, I posted the message below about problems we were
seeing with AdvFS on 5.1A where performance dropped off when there were a
large number of small files (typically less than 8KB). According to Tom
Blinn, we shouldn't have been seeing the problems we were, so the issue
was logged with Compaq.
To cut a long story short, HP have eventually admitted that there is a
problem. To quote support:
"It seems that you can either have AdvFS with a frag file and have the
perf drop at 50% disk usage or with no frag file and get 100% better
perf, with no drop off, but at a cost of 100% more disk usage.
AdvFS engineering's proposed solution to the problem of many small
files
is to remove the frag file altogether and implement variable page size
blocks. However, this will require a new metadata layout on disk and
such a major change would not be delivered as a patch to the current
releases."
When asked about timescales for a fix the response was:
"Since the next release, V5.1B, is already "in the can" and about to be
released any change would have to come with V5.1C in early 2004. It's
on
their radar but all new Tru64 AdvFS features will have to compete for
resources with the AdvFS port to HP-UX."
Consequently we're currently in discussion with HP, regarding getting our
36GB disks swapped out for 72's so that we can actually store the amount
of data that we need to..
Regards,
Rob Leadbeater
-----Original Message-----
From: rob.leadbeater@lynx.co.uk [SMTP:rob.leadbeater@lynx.co.uk]
Sent: Monday, March 18, 2002 4:38 PM
To: tru64-unix-managers@ornl.gov
Subject: Large number of files in AdvFS
Hi managers,
I had hoped that upgrading to 5.1A (from 4.0F) would solve issues that we
had with AdvFS handling lots of small files, however it would appear that
there are still issues, despite what I'm reading in the man pages...
I have a single volume domain, consisting of (currently) 9 file sets. The
volume is 180GB, RAID-5, provided by a HSG80 Fibre channel controller.
Each file set, contains a months worth of data, with the data for each
day being held in a separate directory, and then each day being split up
again into 100 directories, to prevent there being too many files in a
single directory.
So for example, for February i've got:
/images/2002/02/ {01-28} / {00-99}
/images/2002/02 is the mount point for the file set image_r1#2002-02.
Each directory at the bottom of the tree contains approximately 1200
files, with file sizes ranging from 1KB to 10KB.
To populate the data, I'm currently copying the files over from our live
system. As the number of files in the domain increased, performance
dropped off considerably, until the stage, where one of the copy
processes was taking up 98-99% of CPU, making the machine essentially
unusable.
Having had similar issues under 4.0F, which were worked around by
specifying the -x and -p options to mkfdmn, I've had a quick look at the
special files in the .tags directories of each fileset. (output at end)
The large number of extents being shown in M-10 is presumably what is
causing the issue, but how to fix / prevent it ?
Having got the luxury of a second identical 180GB volume to play with, I
tried creating the filesets with the nofrag option set. This appears to
prevent the large number of extents, and I don't get the performance
hits. However for the same number of files I'm using nearly double the
amount of disk space with each file taking up a minimum of 8KB, so this
isn't a long term option.
Could someone give me some hints as to how best to configure things.
Regards,
Rob Leadbeater
# showfile -x M-6
Id Vol PgSz Pages XtntType Segs SegSz I/O Perf File
fffffffa.0000 1 16 74 simple ** ** ftx 7% M-6
extentMap: 1
pageOff pageCnt vol volBlock blockCnt
0 1 1 32 16
1 1 1 154427264 16
2 1 1 167214720 16
3 1 1 337188336 16
4 1 1 192687696 16
5 1 1 227443408 16
6 1 1 331138000 16
<snip>
70 1 1 313102272 16
71 1 1 320029040 16
72 1 1 327788656 16
73 1 1 334896688 16
extentCnt: 74
# showfile -x M-10 | pg
Id Vol PgSz Pages XtntType Segs SegSz I/O Perf File
fffffff6.0000 1 16 568747 simple ** ** ftx 81% M-10
extentMap: 1
pageOff pageCnt vol volBlock blockCnt
0 1 1 48 16
1 1 1 142199920 16
2 463744 1 142208128 7419904
463746 47 1 283798720 752
463793 81 1 149628032 1296
463874 27 1 198649200 432
463901 27 1 209264512 432
463928 27 1 236569312 432
463955 27 1 301503296 432
<snip>
568719 6 1 342612384 96
568725 2 1 342612496 32
568727 2 1 342612544 32
568729 4 1 342612592 64
568733 4 1 342612704 64
568737 10 1 342402816 160
extentCnt: 14525
This message is intended only for the use of the person(s) ("The intended
Recipient(s)") to whom it is addressed. It may contain information which
is privileged and confidential within the meaning of applicable law. If
you are not the intended recipient, please contact the sender as soon as
possible. The views expressed in this communication are not necessarily
those held by LYNX Express Limited.
This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:48:55 EDT