On Wednesday, July 23, 2003, at 10:42 PM, Naveen Kumar Kaushik wrote:
I have non zipped larger than 2 GB file captured by some other tool
(ie
Wiretap and zlib limitation does not come in to picture).
The zlib limitation doesn't come into the picture, as the file isn't
gzipped.
However, the Wiretap limitation does come into the picture, because
you're trying to open the file in Ethereal - *ALL* capture files,
*REGARDLESS* of the tool that captured them, are read using the Wiretap
library code.
Now By design, all versions of Win32 (Win9x/ME/NT/2000) provide full
support
for 64bit pointer file operations ,
Are you certain of that? Windows NT (NT 3.x, NT 4.0, NT 5.0 a/k/a
Windows 2000, NT 5.1 a/k/a Windows XP, NT 5.2(?) a/k/a Windows Server
2003) might support it, but are you certain that Windows OT (95, 98,
me) support it?
so can we just change source so that for
unzipped file of size greater than 2 GB can be opened on window.ie
limited
LFS support.
As you said in your first sentence - and as I explained in my first
reply in this thread - "just chang[ing] the source" isn't necessarily
easy. Wiretap can't use "fseek()", for example, as the offset in
"fseek()" is a "long", and, on Windows *AND* ILP32 UNIXes, a "long" is
32 bits. Unfortunately, the "fsetpos()" alternative uses an "fpos_t",
which is *not* guaranteed to be an integral data type; equally
unfortunately, Wiretap and its callers expect that seek pointers in
files *are* integral data types.
*If* fpos_t is an integral data type on Windows and remains so, and if
it's also an integral data type on those UNIX systems that support it
and remains so, *then* we could, on platforms that support "fsetpos()"
*and* where "fpos_t" is an integral data type greater than 32 bits
*and* where "fopen()" can open a file >2GB and let you seek on a file
>2GB (i.e., where you don't, for example, have "fopen()" not opening
with O_LARGEFILE and O_LARGEFILE being necessary to read past the 2GB
point), support reading capture files greater than 2GB in size
(assuming the capture file format supports it - libpcap format doesn't
have any file offsets in the file, so it should support it, but Network
Monitor format has 32-bit file offsets, so it could, at best, support
files 4GB in size).
On other platforms, however, you're out of luck.
(BTW, the Single UNIX Specification, Version 3, section on "Standard
I/O Streams" seems to imply that C99 requires that an "fpos_t" has to
include some junk to handle "wide-oriented" streams, which suggests
that any system whose C library conforms to C99 will almost certainly
*not* have "fpos_t" be an integral data type. Perhaps there aren't
many systems now that conform to C99, but there might be some in the
future....)
This can possibly extended to Linux since future kernel releases of
linux
might have LFS support (since Linux group is working on it).
*Current* kernel releases of Linux support it; did you not read Ben
Greear's mail, in which he said he had a program that read capture
files >2GB on Linux? Current GNU libc also supports it, as far as I
know.
But Linux is far from being the only interesting non-Windows platform
for Ethereal; it works on all the BSDs, and the BSDs have supported
files >2GB since 4.4BSD, and Solaris, on which Ethereal also runs, has
supported it since Solaris 2.6. Ethereal also runs on Digital UNIX,
but "long" is 64 bits there so it supports files >2GB for free.
Unfortunately, few of them make it easy to support reading those files
with Wiretap the way it currently works. We might change it to do its
own buffering at some point, in which case it might be easier to
support it on Windows and on UNIX platforms that either natively
support 64-bit file offsets (e.g., the BSDs) or support the Large File
Summit APIs, but that's probably not going to happen soon, so,
unfortunately, you'll continue to face the problem of handling very
large capture files for a while.