Wireshark-dev: Re: [Wireshark-dev] pcap-ng support

From: Stephen Donnelly <stephen@xxxxxxxxxx>
Date: Tue, 22 Jan 2008 10:57:47 +1300
On Mon, 2008-01-21 at 22:00 +0100, Ulf Lamping wrote:
> Gianluca Varenni schrieb:
> > I think the description of timestamp formats is quite bad in the specs.
> > The timestamps are represented as a 64bit quantity split into high and low 
> > 32 bits, that represent the number of microseconds/nanoseconds/??? from 
> > 1/1/1970 (that's the meaning of in "in standard unix format i.e. since 
> > 1/1/1970").
> > The reason behind using a single 64bit quantity instead of 
> > seconds/subseconds is twofold:
> > 1. if we use seconds and subseconds, 32bits don't allow to go under the 
> > nanosecond.
> > 2. several hardware-based capture cards represent timestamps as 
> > nanoseconds/microseconds as a single 64bit quantity i.e. they don't split 
> > them into seconds and subseconds.
> >
> > BTW, there was a discussion on the timestamp format on the ntar-workers 
> > mailing list, here
> >
> > http://www.winpcap.org/pipermail/ntar-workers/2006-March/000122.html
> >   
> Yes, the timestamp spec of the EPB (and PB) is *very misleading* here 
> and definitely needs a clarification! The structure - and the 
> descriptive text - looks far too much "libpcap like" to get an idea that 
> it's actually different.
> 
> Reading the text a few times now, I think it's even not very consistent 
> in itself ...

I believe part of the idea behind allowing the timestamp precision to be
specified as a binary fraction (2^-X) is to allow the use of Endace ERF
timestamps in pcapng/ntar natively.

Timestamps in network traces are typically 64-bit, comprising a 32-bit
'seconds' part with a UNIX epoch, and a 32-bit 'subsecond' field
counting milli/micro/nanoseconds withing the second. The most
significant few bits of the subsecond are not used as the conuter
saturates before using all the available bits.

ERF timestamps are also 64-bits, and can be considered to be a 64-bit
value in units of 2^-32 seconds, with a UNIX epoch. Alternatively it can
be considered a fixed point count of seconds from the UNIX epoch.
Conveniently the 32 MSB are equivalent to the conventional 'seconds'
count, while the 32 LSB can be considered a binary fraction of seconds.
The lowest few LSB may not be used by specific hardware depending on the
available clock resolution.

Advantages include the ability to perform a single 64-bit integer
subtraction to find timestamp differences, and the ability to support
different hardware clock resolutions without changing timestamp format
or interpretation.

Using ERF timestamps directly in pcapng avoids the requirement to
convert the timestamp formats on the fly at capture time. This can be a
significant load at extreme packet rates. It does suggest that timestamp
accessor functions should be available so that the any timestamp
(record/block) can be requested in any format.

Stephen.
-- 
-----------------------------------------------------------------------
    Stephen Donnelly BCMS PhD           email: sfd@xxxxxxxxxx
    Endace Technology Ltd               phone: +64 7 839 0540
    Hamilton, New Zealand               cell:  +64 21 1104378
-----------------------------------------------------------------------