Re: NFS file truncating

From: cbaker@GOODYEAR.COM
Date: Thu Feb 12 2004 - 10:14:08 EST


Here is the completed output of "defragfs -r" followed bu a few "defragfs"

> defragfs -r /hpc/light-5
statistics before running defragfs:
number of free fragments 9200753
number of allocated fragments 203316111
number of free spaces shorter than a block 88506
number of free fragments in short free spaces 460607

statistics after running defragfs:
number of free spaces shorter than a block 56651
number of free fragments in short free spaces 378252

other statistics:
number of fragments moved 912128
number of logical blocks moved 298731
number of allocation attempts 67055
number of exact matches 22711

> defragfs /hpc/light-5
statistics before running defragfs:
number of free fragments 9146354
number of allocated fragments 203370510
number of free spaces shorter than a block 88511
number of free fragments in short free spaces 460720

statistics after running defragfs:
number of free spaces shorter than a block 56794
number of free fragments in short free spaces 378493

other statistics:
number of fragments moved 782335
number of logical blocks moved 290520
number of allocation attempts 66536
number of exact matches 22732

> defragfs /hpc/light-5
statistics before running defragfs:
number of free fragments 8634952
number of allocated fragments 203881912
number of free spaces shorter than a block 57272
number of free fragments in short free spaces 380907

statistics after running defragfs:
number of free spaces shorter than a block 53739
number of free fragments in short free spaces 372284

other statistics:
number of fragments moved 99185
number of logical blocks moved 65488
number of allocation attempts 11215
number of exact matches 2401

> defragfs /hpc/light-5
statistics before running defragfs:
number of free fragments 7042121
number of allocated fragments 205474743
number of free spaces shorter than a block 52943
number of free fragments in short free spaces 375240

statistics after running defragfs:
number of free spaces shorter than a block 52575
number of free fragments in short free spaces 373720

other statistics:
number of fragments moved 68578
number of logical blocks moved 27565
number of allocation attempts 4508
number of exact matches 446

Christopher M. Baker
Senior Technical Support Analyst
DSE/TCO
Goodyear Tire and Rubber Company

=================================================
Contains Confidential and/or Proprietary Information.
May not be copied or disseminated without the expressed
written consent of The Goodyear Tire & Rubber Company.
=================================================

                      cbaker@GOODYEAR.
                      COM To: aix-l@Princeton.EDU
                      Sent by: IBM AIX cc: (bcc: Chris Baker/NA/GDYR)
                      Discussion List Subject: Re: NFS file truncating
                      <aix-l@Princeton
                      .EDU>

                      02/11/2004 06:29
                      PM
                      Please respond
                      to IBM AIX
                      Discussion List

/dev/nfs5lv00 1700134912 73600168 96% 337928 3% /hpc/light-5

rdsnfs5:root: /
> defragfs -q /hpc/light-5
statistics before running defragfs:
number of free fragments 9199989
number of allocated fragments 203316875
number of free spaces shorter than a block 88510
number of free fragments in short free spaces 460609

rdsnfs5:root: /
> defragfs -r /hpc/light-5
statistics before running defragfs:
number of free fragments 9200753
number of allocated fragments 203316111
number of free spaces shorter than a block 88506
number of free fragments in short free spaces 460607

{ Hung up on this report..........}

Went ahead and typed in the "defragfs /hpc/light-5" after the report is
done. Had to leave. Will send the results of the report and the defrag
tomorrow.

Thanks,

Christopher M. Baker
Senior Technical Support Analyst
DSE/TCO
Goodyear Tire and Rubber Company

=================================================
Contains Confidential and/or Proprietary Information.
May not be copied or disseminated without the expressed
written consent of The Goodyear Tire & Rubber Company.
=================================================

                      pSeries AIX Geek
                      <aixgeek@YAHOO.C To: aix-l@Princeton.EDU
                      OM> cc: (bcc: Chris
Baker/NA/GDYR)
                      Sent by: IBM AIX Subject: Re: NFS file
truncating
                      Discussion List
                      <aix-l@Princeton
                      .EDU>

                      02/11/2004 05:01
                      PM
                      Please respond
                      to IBM AIX
                      Discussion List

Interesting. Still, I'd suspect fragmentation. If
you run defragfs with the "what-if" option (sorry,
don't have man pages in front of me), what does it
report?

[There's a -q and a -r flag on defragfs. I can't
remember what each does, but the man page for defragfs
will show you what options you have to run to show
what it WOULD do, if you let it.]

Another good test would be to use dd if see if you can
replicate the same error by creating a 4MB+ file on
the server itself (i.e., on the JFS file system, not
on the NFS mount). If you get the same error, I'll
still opine that it's a fragmentation issue.

- pAG

--- cbaker@GOODYEAR.COM wrote:
> No JFS errors in the errpt on the P655. (Actually,
> at the time of the last
> few failed file copies, there have been no errpt
> entries at all.)
>
> Christopher M. Baker
> Senior Technical Support Analyst
> DSE/TCO
> Goodyear Tire and Rubber Company
>
> =================================================
> Contains Confidential and/or Proprietary
> Information.
> May not be copied or disseminated without the
> expressed
> written consent of The Goodyear Tire & Rubber
> Company.
> =================================================
>
>
>
>
> pSeries AIX Geek
> <aixgeek@YAHOO.C To:
> aix-l@Princeton.EDU
> OM> cc:
> (bcc: Chris Baker/NA/GDYR)
> Sent by: IBM AIX
> Subject: Re: NFS file truncating
> Discussion List
> <aix-l@Princeton
> .EDU>
>
>
> 02/11/2004 12:21
> PM
> Please respond
> to IBM AIX
> Discussion List
>
>
>
>
>
>
> Are you getting JFS fragmentation errors in the
> error
> report? On the 4MB boundary, you go from using
> single
> indirection to double indirection and if you don't
> have enough CONTIGUOUS free space, you'll get this
> issue.
>
> Look in your error report for fragmentation errors,
> and look into running defragfs.
>
> - pAG
>
>
> --- cbaker@GOODYEAR.COM wrote:
> > Folks,
> >
> > We have a P655 running as a NFS server. Has been
> > running for months
> > without any problems. But, about a week ago, one
> of
> > our applications
> > started failing when copying files into one of the
> > NFS mounted filesystems.
> >
> > Sometime it works and some times it only partially
> > copies.
> >
> > Strange thing is, many times when it fails, the
> > files that do not
> > completely copy are all the exact same size when
> > they fail. That size is
> > 4,194,304 bytes.
> > In other words, we copy multiple files, the first
> > one is less than 4 MBytes
> > and it copies fine. The next three files are
> bigger
> > than 4 MB, some as big
> > as 600 MB, but each copy gives an I/O error then
> > moves on to the next file.
> > When it is all done, the three larger files are
> each
> > only 4,194,304 bytes
> > big.
> >
> >
> > DETAILS:
> > ========
> >
> > 1) The copy is really a "rcp" running on an HP
> J6000
> > (HP-UX). It is remote
> > copying a file from a third box ( a Tru64 system)
> > back to itself. But it
> > is copying to a filesystem that is the NFS mounted
> > filesystem off of the
> > P655.
> >
> > 2) We see no errors on the Tru64 box, the HP-J6000
> > or the P655.
> >
> > 3) The files being copied are mainly binary files.
> >
> > 4) The copy is done using a rcp that is part of
> LSF
> > (Load Share Facility).
> > It seems to fail with this error:
> >
> > "copyFile: write(802816) failed, No such
> > file or directory."
> >
> > This is followed by: "lsrcp: main()
> > failed, try rcp...."
> >
> > Then it appears to try to use "rcp": and
> that
> > fails also with an
> > error of: "I/O error"
> >
> > 5) Strange thing is, the copy (lsrcp) worked just
> > seconds before on a small
> > ASCII log file (the first file) from and to the
> same
> > directories??
> >
> > 6) This problem started about the same time we
> > increased the load on the
> > NFS server. But, system does not seem to be
> > anywhere near loaded.
> >
> > 7) The filesystem in question resides on EMC
> > Clariion disks that are
> > attached via fiber channel to the P655.
> >
> > 8) AIX 5.1 ML04 on NFS server.
> >
> >
> > Any help, suggestion or sly remarks would be
> greatly
> > appreciated.
> >
> > Thanks,
> >
> > Christopher M. Baker
> > Senior Technical Support Analyst
> > DSE/TCO
> > Goodyear Tire and Rubber Company
> >
> > =================================================
> > Contains Confidential and/or Proprietary
> > Information.
> > May not be copied or disseminated without the
> > expressed
> > written consent of The Goodyear Tire & Rubber
> > Company.
> > =================================================
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Finance: Get your refund fast by filing
> online.
> http://taxes.yahoo.com/filing.html

__________________________________
Do you Yahoo!?
Yahoo! Finance: Get your refund fast by filing online.
http://taxes.yahoo.com/filing.html



This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:17:35 EDT