SUMMARY: Maximum number of files that can be copied or moved

From: Maglinger, Paul (PMAGLINGER@scvl.com)
Date: Mon Oct 21 2002 - 10:28:55 EDT


> Can anyone tell if there is a maximum number of files that can be copied
or
> moved using wildcards in Tru64 Unix 5.1a at one time?
>
>

My thanks to Dr. Blinn, Mike Smith, Mandell Degerness, Jim Belonis, Cory
Erickson, Tim Cutts, Martin Adolfsson, and Joerg Bruehe for their responses.

Some of the responses are as follows:

>From Dr. Blinn:

The practical upper limit is based on the number of characters
of arguments you can pass from the shell (which is what parse
the command, including expanding wildcards) and the command you
are invoking. That number is a UNIX standard, and as I recall
it's something like 2048 bytes. Obviously, with longer names
you can pass fewer of them. From a practical perspective, you
can use "xargs" to break up a list of names and pass them to
the appropriate command, along with the appropriate target, or
you can use a utility such as "tar" or "pax" that can accept a
source file list, which works quite well for copies (and less
well for moves, since you typically have to remove the files
yourself, although as I recall "pax" has a move mode as well
as a copy mode).

>From Dr. Cutts:

In most UNIX variants, this is a property partly of the shell, and
partly of the operating system.

Usually there is a maximum command length, defined by the operating
system. I don't know how big this is for Tru64. Then there's also
likely to be a maximum command line length in the shell you're using,
which may or may not be smaller than the Tru64 limit. Typically system
shells (sh, csh) have smaller limits than more recent shells you might
use (such as bash).

Usually, there are other, better ways of copying large numbers of files
which get around the limit. Try some of these:

tar cf - directory | ( cd target_dir && tar xf - )

or

rsync -a directory target_dir

or use find, although that's usually a bit less efficient than the above
methods, and may not preserve ownerships and permissions.

For many commands, xargs can usually be used to automatically build
commands that fit in the command line length, and execute them. For
example:

cat /etc/passwd | cut -d: -f1 | xargs -l1 finger

will run finger for every user in your passwd file, even if you've got
thousands of them. I don't recommend you do this for real. :-)

>From Cory Erickson:

You are limited to ~38k of command line. This is hard set in sys/limits.h
(ARG_MAX).

Thanks everybody!



This archive was generated by hypermail 2.1.7 : Sat Apr 12 2008 - 10:48:57 EDT