Jakub Zawadzki wrote:
On Wed, Nov 03, 2010 at 03:00:10PM -0400, Jeff Morriss wrote:
Memory mapped files count as part of the process' address space so
mapping the capture file means that, for reassembly, we'd be trading
malloc'd memory for mmap()'d memory *and* it would mean we're holding
the whole file--including the bits not used in reassembly--in memory.
(We don't currently do this.)
Well I'm not expert in mmap()'ed files, but afair they won't be read until page fault,
They may not be loaded into memory, but AFAIK the mapped memory will
immediately count towards the process' (used) address space.
and madvise(..., MADV_DONTNEED) should free memory.
Never heard of that until now; interesting... But DONTNEED just says "I
won't need it soon" (so feel free to page it out--but that won't reduce
our used address space). REMOVE would mean "unmap the memory" but the
Linux man page says that only works with shmfs/tmpfs--and anyway we will
probably need it again. And I'm guessing it's not portable (the Linux
man page makes a reference to Posix 2001).
Still it's lot easier to work with than fread() + fseek()
Memory-mapped files also are problematic when doing live captures: as
the file grows we'd have to remap the memory (yikes!)
Ok, I agree this is problem. But why yikes?
If we attempt to remap we're not guaranteed that the mapping will land
at the same address again. If not, all our pointers into the map are
now invalid. There may be other problems too (if that's not enough ;-)).