Re: speeding up tar

From: Ian Smith (ian.smith@COMPUTING-SERVICES.OXFORD.AC.UK)
Date: Thu Apr 10 2003 - 06:42:46 EDT


John,

You can try the following - the tar reads the output of find on STDIN :

tar -cf FILE.TAR `find . -name "*.txt"`

this should be quicker but only works, however, if none of the files have
spaces in them.

If all the files are under one directory then you can just use:

tar -cf FILE.TAR *.txt

If all the files are under one directory and it's next level sub-directories
you can use:

tar -cf FILE.TAR *.txt */*.txt

I don't know how to code the file spec for files in directories 2 or more
levels under the current dir though. Anybody ?

However, if I was tarring up 30000 files I'd want to make just a cursory
check at least, that all the files I thought were archived, had indeed been
archived. In which instance I'd probably do a tar -tvf FILE.TAR | wc -l
and compare that with the find command piped through wc -l. Probably not
all that quick in the end !

-------------------------------------------------------------------
Ian Smith
Oxford University Computing Services, Oxford, UK.
-------------------------------------------------------------------

~>MIME-Version: 1.0
~>X-Spam-Level: *
~>X-Spam-KB: http://www.Princeton.EDU/spam
~>X-Spam-Report: CARRIAGE_RETURNS,EXCHANGE_SERVER,SPAM_PHRASE_00_01
~>Date: Thu, 10 Apr 2003 11:04:43 +0100
~>From: John Dunn <john.dunn@SEFAS.CO.UK>
~>Subject: speeding up tar
~>X-To: AIX List <AIX-L@PUCC.PRINCETON.EDU>
~>To: aix-l@Princeton.EDU
~>
~>I have a script that tars up lots of files....up to 30,000
~>
~>Because of the number of files and the need to avoid the "arg list too long"
~>problem, I am running a find command to list the files
~>
~>find . -name "*.txt" > tarlist
~>
~>tar -cf$TARFILE -L tarlist
~>
~>I am wondering whether using the find and then the -L method is relatively
~>slow and whether avoiding using a list would speed things up.
~>
~>How can I tar up 30000 files without using find and -L??
~>
~>John



This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:16:44 EDT