Wireshark-dev: Re: [Wireshark-dev] one possible way to speed up filtering

Date: Thu, 19 Mar 2009 07:48:40 -0400
yami wrote:
Thanks, I've written a page in Wiki:
  http://wiki.wireshark.org/Development/FastFiltering


On a side note, I've had an idea brewing that's along the same lines that I'll throw out there for comment.

I often work with fairly large capture files (>100MB) and running filter after filter gets time consuming.

Why not change filtering / packet processing to an "on demand" feature? eg. I create a display filter and apply it. WS then starts at the current packet position and applies the filter forward and backwards until it fills the packet list pane with matched packets. As I move forwards or backwards through the packet list pane, WS would process a little more of the file and have a reasonable amount buffered (say one or two panes full).

If this were carried one step further, and processed packets were freed once they fell off the packet list pane, huge (multi-gigabyte?) files could be processed. A delay would maintain a reasonable buffer in case the user changed direction.

I'll admit I'm confusing packet processing with packet filtering, but I think you get the idea.

In the event of reading in a huge file (hundreds of MB to multi GB), Just process, say, three times the number of packets viewable in the current packet list pane.

For those concerned about overall stats (like at the bottom of the window: displayed, filtered, etc), once the above mentioned buffer is filled, return control to the user and continue processing in the background at low prio. A subtle way to let the user know WS is still applying the filter would be to change "Displayed: 34532" to "Displayed: 35%" until WS finishes and has an exact number. The percentage could be either percent processed (100% = finished), or running percentage of packets processed which match the display filter.

Well, I think I've rambled on long enough.  Thoughts?

Jason.