From: Dave.Zarnoch@SUNGARD.COM
Date: Thu Aug 08 2002 - 15:02:39 EDT
Bill,
I found this on the "net":
VERY LONG......
Dave Zarnoch
UNIX Systems Administration
SunGard eSourcing
600 Laurel Oak Rd.
Voorhees, NJ 08043
Dave.Zarnoch@sungard.com
Code from src/create.c
if (dereference_option != 0
#ifdef STX_HIDDEN /* AIX */
? statx (p, ¤t_stat, STATSIZE, STX_HIDDEN)
: statx (p, ¤t_stat, STATSIZE, STX_HIDDEN | STX_LINK)
#else
? stat (p, ¤t_stat) : lstat (p, ¤t_stat)
#endif
)
{
WARN ((0, errno, _("Cannot add file %s"), p));
if (!ignore_failed_read_option)
exit_status = TAREXIT_FAILURE;
return;
}
Error message:
Jupiter:/cool/ctload/eds #>ls -ln alden_ram.unl.gz
-rw-r----- 1 4002 402 3711884961 Feb 18 04:18 alden_ram.unl.gz
Jupiter:/cool/ctload/eds #>gtar -cf /dev/rmt0 alden_ram.unl.gz
gtar: Cannot add file alden_ram.unl.gz: The user ID or group ID is to large to f
it in the provided structure.
gtar: Error exit delayed from previous errors
Jupiter:/cool/ctload/eds #>
This is from their web docs. Hope it helps.
Writing Programs Which Access Large Files
Beginning in Version 4.2, the operating system allows files which are larger than 2 gigabytes
(2GB). This article is intended to assist programmers in understanding the implications of
"large" files on their applications and to assist them in modifying their applications, if they
choose, to be aware of large files.
The file system programming interfaces generally revolve around the off_t data type. In
Version 4.1, the off_t data type was defined as a signed 32-bit integer. As a result, the
maximum file size that these interfaces would allow was 2 gigabytes minus 1.
A new set of programming interfaces is defined, so that application programs can be aware of
large files.
Implications for Existing Programs
The 32-bit application environment which all applications used in prior releases remains
unchanged. Existing application programs will execute exactly as they did before. However,
existing application programs will not be able to deal with large files.
For example, the st_size field in the stat structure, which is used to return file sizes, is a
signed, 32-bit long. Therefore, that stat structure cannot be used to return file sizes which are
larger than LONG_MAX. If an application attempts to stat a file which is larger than
LONG_MAX, the stat subroutine will fail, and errno will be set to EOVERFLOW, indicating
that the file size overflows the size field of the structure being used by the program.
This behavior is significant because existing programs which might not appear to have any
impacts as a result of large files will experience failures in the presence of large files even
though they may not even be interested in the file size.
The errno EOVERFLOW can also be returned by lseek and by fcntl if the values which need
to be returned are larger than the data type or structure which the program is using. For lseek,
if the resulting offset is larger than LONG_MAX, lseek will fail and errno will be set to
EOVERFLOW. For fcntl, if the caller uses F_GETLK and the blocking lock`s starting offset
or length is larger than LONG_MAX, the fcntl call will fail, and errno will be set to
EOVERFLOW.
Open Protection
Many of the existing application programs were written under the assumption that a file size
could never be larger than could be represented in a signed, 32-bit long. These programs
could have unexpected behavior, including data corruption, if allowed to operate on large
files. Beginning in Version 4.2, the operating system implements an open-protection scheme to
protect applications from this class of failure.
When an application which has not been enabled for large-file access attempts to open a file
which is larger than LONG_MAX, the open subroutine will fail and errno will be set to
EOVERFLOW. Application programs which have not been enabled will be unable to access
a large file, and the possibility of inadvertent data corruption is avoided. Applications which
need to be able to open large files must be ported to the large-file environment described in
"Porting Applications to the Large File Environment".
In addition to open protection, a number of other subroutines offer protection by providing an
execution environment which is identical to the environment under which these programs
were developed. If an application uses the write family of subroutines and the write request
crosses the 2 gigabyte boundary, the write subroutines will transfer data only up to 2
gigabytes minus 1. If the application attempts to write at or beyond the 2Gb-1 boundary, the
write subroutines will fail and set errno to EFBIG. The behavior of mmap, ftruncate, and
fclear are similar.
The read family of subroutines also participates in the open protection scheme. If an
application attempts to read a file across the 2 gigabyte threshold, only the data up to 2
gigabytes minus 1 will be read. Reads at or beyond the 2Gb-1 boundary will fail, and errno
will be set to EOVERFLOW.
Open protection is implemented by a flag associated with an open file description. The
current state of the flag can be queried with the fcntl subroutine using the F_GETFL
command. The flag can be modified with the fcntl subroutine using the F_SETFL command.
Since open file descriptions are inherited across exec family of subroutines, application
programs which pass file descriptors which are enabled for large-file access to other
programs should consider whether the receiving program can safely access the large file.
Porting Applications to the Large File Environment
Beginning in Version 4.2, the operating system provides two different ways for applications
to be enabled for large-file access. Application programmers must decide which approach
best suits their needs. The first approach is to define _LARGE_FILES, which carefully
redefines all of the relevant data types, structures, and subroutine names to their large-file
enabled counterparts. The second approach is to recode the application to call the large-file
enabled subroutines explicitly.
Defining _LARGE_FILES has the advantage of maximizing application portability to other
platforms since the application is still written to the normal POSIX and XPG interfaces. It has
the disadvantage of creating some ambiguity in the code since the size of the various data
items is not obvious from looking at the code.
Recoding the application has the obvious disadvantages of requiring more effort and reducing
application portability. It can be used when the redefinition effect of _LARGE_FILES would
have a considerable negative impact on the program or when it is desirable to convert only a
very small portion of the program.
It is very important to understand that in either case, the application program MUST be
carefully audited to ensure correct behavior in the new environment. Some of the common
programming pitfalls are discussed in "Common Pitfalls Using the Large File Environment".
Using _LARGE_FILES
In the default compilation environment, the off_t data type is defined as a signed, 32-bit long.
Beginning in Version 4.2, if the application defines _LARGE_FILES before the inclusion of
any header files, then the large-file programming environment is enabled. and off_t is defined
to be a signed, 64-bit long long. In addition, all of the subroutines which deal with file sizes
or file offsets are redefined to be their large-file enabled counterparts. Similarly, all of the
data structures with embedded file sizes or offsets are redefined.
Assuming that the application is coded without any dependencies on off_t being a 32-bit
quantity, the resulting binary should work properly in the new environment.
Bill Verzal
<Bill_Verzal@BCBS To: aix-l@Princeton.EDU
IL.COM> cc:
Sent by: IBM AIX Subject: "cp" error message
Discussion List
<aix-l@Princeton.
EDU>
08/08/2002 04:33
PM
Please respond to
IBM AIX
Discussion List
cp: LPARBkups/siebdapp/siebdappvg_20020801_135657: The user ID or group ID
is to large to fit in the provided structure.
huh ? Anyone seen this before ?
-----------------------------------------------------------------------------------------------------------
Bill Verzal
Technical Consultant
Forbes Technical Consulting
(312) 653-3684
bill_verzal@bcbsil.com
MailStop: 27.201C
This archive was generated by hypermail 2.1.7 : Wed Apr 09 2008 - 22:16:08 EDT