["Farley, Tim (ISSAtlanta)" <TFarley@xxxxxxx>]

>There appear, at least in the compressed Sniffer file in 
>question, to be *three* records at the beginning of the file, 
>*none* of which are compressed...

That is correct, Sniffer ONLY compresses the actual frame data.  The
"header" records that start the file are NEVER compressed.  However, DO NOT
make the assumption that there will only be three of these, or they will be
a certain length, or of certain types.  Various versions of Sniffer put
different combinations of records up front.  That's why the Sniffer format
has these little chunk headers on them, so they could mutate the file format
subtly without breaking forward and backward compatibility.  For instance,
if a new version of Sniffer needs to store a new header item, it just does
so in a new chunk type which all other implementations will simply ignore.

The logic I use in my program here at ISS which reads Sniffer captures is as
follows.  Start reading the file normally, until you get the header record
that tells you the file is going to be compressed.  Just record this fact,
but don't do anything with it immediately.

Continue to read & skip over records which appear to be of the normal header
types, until such time as you read a header record that has a "type" field
that is outside of the range of 0 to 16 or so.  If you earlier saw in the
header that this file is compressed, then start up the compression code and
start treating the rest of the file as compressed blobs.  If you didn't get
a compressed flag in the header, then you can either decide to proceed as if
the file is compressed and hope the header was wrong, or error out and
announce that the file is corrupt.

This takes advantage of the fact that the "length" word (a signed 16-bit
integer) of a compressed blob is the first two octets of the blob, but the
"type" field of a non-compressed frame record is also the first two octets
of the record.  So these two items will appear at the same spot in the file.
However, their valid ranges do not overlap.  The "type" field always
contains small integers like 4 and 6, but a compressed block would NEVER
have a length this small.

>	1) Can we really guarantee that no compressed blob expand to
>	   more than 65536 bytes?  If not, we may have to reallocate the
>	   output buffer if we don't have room to uncompress a blob.

Yes.  In fact, no compressed blob every expands to more than 16,384 bytes in
any file I have ever seen.  It appears that the 16K buffer is statically
allocated in some implementations of the decompression algorithm I have
examined.  Architecturally, the theoretical max size of a compressed blob is
32K.  See next comment.

>	2) The code treats a blob with a length of 49152 as being an
>	   uncompressed blob with a length of 16384; 49152 is 0xC000,
>	   and 16384 is 0x4000 - is the blob length really a flag of
>	   0x8000, which if set means the blob is uncompressed, and a
>	   15-bit blob-size field?

No, that's not quite right.  The way it works is the two bytes in front of a
compressed blob is a SIGNED INTEGER (in Intel order of course).  If the
integer is positive, then it is the true length of the compressed blob.
However, if the integer is negative, then the blob is uncompressed and the
ABSOLUTE VALUE of the integer is the length of the uncompressed text.

Note that the fact that a heavily encrypted capture always gets chunked into
uncompressed blobs of 16K in length verifies that 16K is the maximum buffer
size needed.

However it is important that you treat the length just as I describe above,
because you CAN get an uncompressed chunk that is shorter than 16K,
typically at the end of a capture.  If you want a sample file that
demonstrates this let me know.


I'm sorry I didn't submit code that included all this, but the code I wrote
is in C++ and therefore wouldn't have been appropriate for integration with
Ethereal.

=====================================
Tim Farley
X-Force Researcher
mailto:tfarley@xxxxxxx
Direct: (678) 443-6189 / fax (678) 443-6479
=====================================