[bugzilla-daemon@xxxxxxxxxxxxx]

Guy Harris changed bug 12526

What	Removed	Added
Priority	Low	High
Status	UNCONFIRMED	CONFIRMED
Ever confirmed		1
Severity	Major	Enhancement

Comment # 5 on bug 12526 from Guy Harris

(In reply to Anthony Coddington from comment #4)
> (In reply to Guy Harris from comment #2)
> > (In reply to Anthony Coddington from comment #1) 
> > > Does still calling it caplen make sense?
> > 
> > It's called "Captured Packet Length" in the pcapng specification, so I guess
> > we could continue to call it caplen.
> 
> Right, I've always found it a little confusing the caplen includes non on
> the wire pseudoheaders, but here we are effectively matching up the concepts
> again between pcapng and Wireshark.

That dates back to pcap:

    struct pcap_pkthdr {
            struct timeval ts;      /* time stamp */
            bpf_u_int32 caplen;     /* length of portion present */
            bpf_u_int32 len;        /* length this packet (off wire) */
    };

which was created in an era when packet data began with an Ethernet header, a
PPP header, a DLT_NULL header on lo0, an IP header for DLT_RAW, and *maybe* a
Token Ring header, i.e. there weren't any pseudo-headers.

I'm not sure whether that fell apart with ATM or 802.11 or some other network
type, but the path of least resistance, at that point, was to just stuff
metadata before the actual link-layer header (or whatever you want to call it
for ATM AAL5 PDUs) and not make complicated changes to BPF and DLPI and
PF_PACKET sockets and so on to separate out metadata and packet data.

Sadly, that opened up the issue of "OK, should the a snapshot length of N chop
the metadata+packet data blob N bytes into the entire blob, or should it always
give you all the metadata and chop it N bytes into the packet data?"

*If* you're using BPF filter programs, and *if* the snapshot length is
implemented the way the BPF capture mechanism implements it, with the return
value of the BPF program, if non-zero, being the length at which to chop the
packet, *then* it's possible, at least of the 802.11 radio headers, to
implement "chop it N bytes into the packet data", by adding N to the
pseudo-header length, putting the result in the BPF machine's accumulator, and
doing a "RET A" instruction.  That's probably what *should* have been done, but
we were all lazy and didn't do that, and changing it now *might* disrupt
existing scripts etc..  If the disruption would be non-existent, minimal, or
pretty much harmless as a result of providing *more* data, maybe we ("we" the
libpcap developers, probably ending up meaning "me") should do that.

I should probably raise that question on tcpdump-workers.

> 
> > > What about trailing 'pseudoheaders' like PCAP-NG EPB options?
> > 
> > The libwiretap API should, for all records it returns, provide an array of
> > options, in the same form as the array of options that the wtap_opttypes
> > code does.  The pcapng EPB options should be supplied in that fashion,
> > rather than as payload.
> > 
> > And, yes, that's probably how the ERF reader should handle its extension
> > headers as well.
> > 
> > > Is it
> > > reasonable to assume they all belong to one dissector that passes an
> > > appropriate tvb downstream or does more than one dissector need to handle
> > > the pseudo header?
> > 
> > They wouldn't even be *in* a tvbuff.
> > 
> > > Things like CPacket trailer timestamps are even more murky since they affect
> > > the Ethernet CRC, I'd probably argue those shouldn't count as phdr_len but
> > > am not sure. Otherwise dissectors might need to be able to dynamically
> > > update phdr_len which might get messy, as wiretap may not be fully aware of
> > > possible pseudoheadrs in that case. Perhaps the definition here should be it
> > > is only pseudoheader if it relates to a wtap_encap encapsulation type?
> > 
> > It's a pseudo-header if it relates to a file type or encapsulation type (and
> > that may mean we'd supply two pseudo-headers in some cases).
> 
> There are two related but different ideas here: I understand the idea
> currently proposed in this RFC is to put ALL captured data as part of
> 'payload' and passing them to regular encap dissectors for display in the
> tree similar to the USB dissector (possibly even including file level
> information) as just another 'layer' in the packet.

Well, there's "what does libwiretap do?" and there's "what does libwireshark
do?"

Wiretap could just provide a single blob with metadata and packet data, along
with an indication of how big the metadata is; libwireshark could chop it into
multiple tvbuffs.

> Some metadata might need
> to also be extracted at the wtap level e.g. optionblocks but the phdr union
> becomes scrapped in concept.

I haven't *quite* figured out what the right way is to handle metadata that's
1) not the sort captured by pcapng options and 2) not supplied as part of the
packet data.

To take one of the biggest users of metadata, 802.11:

    The various "802.11+radio" link-layer header types for pcap/pcapng provide
radio metadata as part of the packet data, just because the capture mechanisms
and the pcap file format, and the libpcap API, don't provide for separating
them.

    Microsoft Network Monitor does the same with the radio metadata; I don't
know about the capture mechanism and the APIs, which Microsoft control and
don't export as far as I know (well, there might have been some APIs for the
capture mechanism), but the file format didn't provide for separating them
(probably for the same reason as pcap - it predated 802.11).

    The "classic" flavor of the *Peek format from WildPackets^WSavvius does the
same, possibly for the same reason - I think the first product in that line was
EtherPeek, followed by TokenPeek, with AiroPeek coming later.

    The "tagged" flavor of *Peek format, however, provides the radio metadata
as part of a TLV list of metadata headers, rather than as part of the packet
data.

    Shomiti^WFinisar's Surveyor also provides it out of band; they could get
away with that because they based their format on Sun's snoop format, which
nominally doesn't support metadata outside the packet data, *but* which pads
all records to a 4-byte boundary *and* has a separate "record size" that
includes the padding, rather than by implicitly padding to the *next* 4-byte
boundary, so you can - and Shomiti did - sneak in more "padding" and put actual
metadata into it.

    Tamosoft's CommView Wi-Fi, Network Instruments Observer, and Network
Associates, err, umm, Network General, err, umm, NetScout's Sniffer also
provide it out of band.

That's why wiretap's handling of radio metadata is a bit weird.  For all the
"out of band" flavors, the WTAP_ENCAP_ value is
WTAP_ENCAP_IEEE_802_11_WITH_RADIO, and a struct ieee_802_11_phdr, filled in as
appropriate is provided as the pseudo-header.  That's also done for *Peek
classic. For all the "in band" flavors, except for *Peek classic, we supply the
metadata as is, assign a WTAP_ENCAP_ value to it, and expect the dissector to
construct a struct ieee_802_11_phdr and pass it to the 802.11 radio dissector.

(Heading off to meet somebody for dinner; I'll continue this in the next
comment.  Confirming, and classifying as an "enhancement", as the behavior
might be wrong - and I'd say it is - but it's not as if we "intended" something
else.  Also raising the priority.)

---

      You are receiving this mail because:

• You are watching all bug changes.