From: Bill Thompson (bill.thompson@GOODYEAR.COM)
Date: Wed May 05 2004 - 07:07:17 EDT
> I assume you didn't try xargs.
You assume correctly. While xargs can be a great help in reducing the number of processes you have to spawn it is inapplicable in this
case. The need is to convert each filename to uppercase. The generic usage was "UPPER=<conversion method>" (e.g.: UPPER=$(echo "$FILE"
| tr -s "a-z" "A-Z")). There is no way to use xargs in this context.
But the point really was to demonstrate how costly it is to spawn a new process and xargs is a great tool to help reduce this
overhead. Here's an example:
I ran a find against the /usr filesystem on one of my AIX servers to get a long listing of all the files (ignoring find's -ls option).
I ran it once using find's -exec option and again by piping the output of find into xargs. Note that the -exec option of find spawns
another process for *each* file found while xargs processes multiple files in a single spawned process.
This first example took 4 minutes 43 seconds to run.
find /usr -exec ls -ld {} \;
This second example using xargs took 0 minutes 9 seconds to run
find /usr -print | xargs ls -ld
Not as dramatic a difference but if you're writing a script it means the difference in a user having to wait a few seconds for output
as compared to almost five minutes.
Bill Thompson
Sr UNIX Systems Administrator
The Goodyear Tire & Rubber Co.
Contains Confidential and/or Proprietary Information
May Not Be Copied or Disseminated Without Express Consent of The Goodyear Tire & Rubber Company.
AIX-L Archives: http://marc.theaimsgroup.com/?l=aix-l&r=1&w=2
----- Original Message -----
From: "Michael Cheselka" <cheselka@LINUX.CACTUS.ORG>
Newsgroups: bit.listserv.aix-l
To: <aix-l@Princeton.EDU>
Sent: Tuesday, May 04, 2004 6:42 PM
Subject: Re: Filename conversion to uppercase
> I assume you didn't try xargs.
>
> On Tue, May 04, 2004 at 07:19:06AM -0400, Bill Thompson wrote:
> > This is an FYI for all of you shell scripters out there.
> >
> > All the methods presented to convert a text string to uppercase characters work equally well however the time they take to do the
> > conversion varies greatly. If you're converting a handful of files you'll never see the difference. However, if you're converting
a
> > large number of files the time difference can be astonishing.
> *snip*
> > I ran these five scripts against a directory containing 84,600 files and recorded the execution time. They were dramatically
> > different:
> >
> > script 1: 8.0 seconds
> >
> > script 2: 2:03:14.5 = 7394.5 seconds
> >
> > script 3: 1:52:44.2 = 6764.2 seconds
> >
> > script 4: 1:52:41.9 = 6761.9 seconds
> >
> > script 5: 2:01:38.1 = 7298.1 seconds
> >
> > No, that's not a mistake. Script 1 took 8 seconds to run while the others took 2 hours. The scripts that spawned another process
(or
> > actually spawned 84,600 processes - one for each filename) took close to 1,000 times longer to perform the same function!
> >
> > So, when you're writing your scripts out there, remember - spawning another process is the most costly thing you can do on a Unix
> > system.
> >
> > Bill Thompson
> > Sr UNIX Systems Administrator
> > The Goodyear Tire & Rubber Co.
> >
> > Contains Confidential and/or Proprietary Information
> > May Not Be Copied or Disseminated Without Express Consent of The Goodyear Tire & Rubber Company.
> --
> Michael R. M. Cheselka ryoohki@ryoohki.org
> Itsu Made Mo "Love & Peace" ryoohki@spymac.com
> http://www.cactus.org/~cheselka cheselka@cactus.org
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:17:53 EDT