[Evan Huus <eapache@xxxxxxxxx>]

On Thu, Aug 29, 2013 at 5:32 AM, Dario Lombardo <dario.lombardo.ml@xxxxxxxxx> wrote:

On Wed, Aug 28, 2013 at 3:45 PM, Evan Huus <eapache@xxxxxxxxx> wrote:

Also, just as a useful reference point: if you run your normal tshark command (no valgrind) on the 1GB subset, what does memory usage of the process peak at?

It uses about 1,5GB of memory.

With the env vars you asked me to export, tshark can't run on the 1GB pcap file. I've reduced the dataset to 2 millions of pkts, and that's the output.

That's quite useful actually: it looks like the majority of the memory is being used to store address-resolution data from all of the DNS packets so that if those IP addresses show up later we can resolve them immediately (without having to ask the system name resolver).

It doesn't look like there's a way to disable this at the moment (I believe we still store the names even if name resolution is disabled), but it should be easy enough to fix. The add_ipv4_name and add_ipv6_name functions should probably be no-ops if all name resolution is disabled. Then simply passing the -n flag will greatly reduce your memory usage (though it won't yet).

If nobody has any objections or better ideas I will probably fix this on the weekend.

Cheers,
Evan