Wireshark-dev: Re: [Wireshark-dev] New packet list - out of memory?

From: "Anders Broman" <a.broman@xxxxxxxxx>
Date: Tue, 6 Oct 2009 20:10:53 +0200

-----Ursprungligt meddelande-----
Från: wireshark-dev-bounces@xxxxxxxxxxxxx
[mailto:wireshark-dev-bounces@xxxxxxxxxxxxx] För Jeff Morriss
Skickat: den 6 oktober 2009 18:21
Till: Developer support list for Wireshark
Ämne: Re: [Wireshark-dev] New packet list - out of memory?

Anders Broman wrote:
>> Hi,
>> Experimenting loading the file there is 163 calls to VirtualAlloc()
>> ~1.630 GB memory used
>> Changing 
>> #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 512)
>> to:
>> #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 64)
>> 
>> Gives only 26 VirtualAlloc() so it seems like we waist a huge amount of
>> memory...
>> 15 VirtualAlloc() with 32 bytes.
>> 
>> Is there a reason not to change to:
>> #define EMEM_ALLOCS_PER_CHUNK (EMEM_PACKET_CHUNK_SIZE / 32) ?
>
>Well, the disadvantage is that the overhead associated with the chunk 
>headers grows (when using canaries).  For example, with the current /512 
>  number sizeof(emem_canary_t) is 184,328 which means 1.47% of the total 
>allocation for the chunk goes to overhead.  Changing that to /64 means 
>the canary header grows to 1,474,568 and total overhead grows to 14.1%. 
>  Those numbers are not counting the actual canaries at the end of each 
>allocation.
>
>BUT, the waste that goes into those headers is vastly overshadowed by 
>the waste in unused buffer space...
>
>> Loading is actually 2s slower with se_alloc for col_text...
>> Are we better off using glibs memory functions?
>
>Another question: do we really need to use se_ allocations here?  Do we 
>need the canaries or are we just using se_ allocations because they get 
>cleaned up for us "automatically?
>
I think the rationale for using se_alloc was a belief that it would be
Faster and of course the ease of use (no free).

>"
>
>Should we have a new allocator without the canaries?
I think either the current se_ allocations implementation need to be changed
Or the function removed. The result where WIRESHARK_DEBUG_SE_NO_CHUNKS=T
And loading is faster is puzzling.
Regards
Anders
___________________________________________________________________________
Sent via:    Wireshark-dev mailing list <wireshark-dev@xxxxxxxxxxxxx>
Archives:    http://www.wireshark.org/lists/wireshark-dev
Unsubscribe: https://wireshark.org/mailman/options/wireshark-dev
             mailto:wireshark-dev-request@xxxxxxxxxxxxx?subject=unsubscribe