Wireshark-dev: Re: [Wireshark-dev] Memory consumption in tshark

From: Anders Broman <a.broman@xxxxxxxxxxxx>
Date: Wed, 28 Aug 2013 00:02:44 +0200
Joerg Mayer skrev 2013-08-27 23:24:
On Tue, Aug 27, 2013 at 05:09:19PM -0400, Evan Huus wrote:
IIRC, two-pass allows for most/all of the reassembly/request-response
stuff,
which we want to do sometimes. Any ideas why we have to keep some
information
indefinitely?

Two-pass requires us to keep *all* the state around through the first pass
so that it is available during the second pass (at which point it can be
discarded).  Even in single-pass mode, there is some state that we can't
always immediately discard. If I see a fragment of a TCP message then it
doesn't make sense to discard that until the other fragments have arrived
and been reassembled. If I see a request, I probably need to keep state
from that request until the response (which may never show up).

We already do reassembly and a lot of other stateful work in single-pass
mode. The only thing two-pass mode provides is the ability to "see the
future" (for example, saying: this request has a response 5 packets later).
So (assuming we really free everything we could already) could add a
possibly configurable foresight horizon of 10000 packets. If a packet
number is older than 10000 packets, forget it?


I haven't really looked but we do keep the reassembled fragments around even in tshark as there is no mechanism to discard them selectively if run by tshark as opposed to wireshark and that's the really big memory eater I would think.
Regards
Anders