[Ian Schorr <spamcontrol2@xxxxxxxxxxx>]

Guy,

You're right, in this case I was using libpcap 0.6, and after loading 
the latest CVS version tcpdump appears to be reporting kernel drops.  
(Incidentally, I had gotten the "kernel" idea because I thought my 
version of libpcap was up to snuff, and I'd remembered a post you'd made 
on the libpcap mailing list where I thought you'd mentioned that even 
the 2.4.x kernels needed to be compiled with the right options to enable 
drop reporting).

"tethereal --version" still reports using 0.6 after installing the 
latest cvs libpcap, and I did't know how to update this - though I'll 
try what you're recommending tomorrow.

Here's what I found today, however:

I'm having several problems that confused the issue.

- I'm bottlenecking on my disk when doing to-disk writes since it looks 
like my disk controller driver is putting all my drives in PIO modes, so 
I was dropping a lot during my save-to-disk tests at least partly for 
this reason
- While running "tcpdump -s 9000 -w /dev/null" I was able to 
receive/process packets at no more than about a 1-2% drop rate until I 
threw about 80MB/s (80,000 1000byte frames) at the system and the CPU 
started maxing out.
- Similarly, with tethereal it appeared that I was seeing the same drop 
rates while capturing data flowing at up to 40MB/s (again, sending to 
/dev/null).  Tethereal wasn't reporting drops, and I'm basing this 
solely on reported received frames versus the total number of frames sent.
- To-file capture with tethereal seemed to be fairly solid (~1% drops) 
up until I started hitting my disk bottleneck (at about 15MB/s).
- However, when I enabled ring-buffer capture (at anywhere between 2-10 
files), drop rates skyrocketed to 30-70%, at rates anywhere between 
1MB/s-12MB/s (1000-12,000 frames per second).  This was true even when 
waiting at least 30 seconds after starting capture to begin generating 
traffic, and even if the total amount of traffic would be saved in only 
one file.  I'm not sure what's causing this.  CPU seems to hang at 25% 
during these tests.
- I also noticed that specifying -a filesize:5000000 (without ring 
buffer mode) seems to cause tethereal to stop capturing after 700M data 
has been received.  This doesn't happen if I set filesize to 1000000 or 
2000000.  This is yet another quirk that probably confused my testing 
yesterday.  Perhaps a bug?
- Ethereal (GUI) capture rates are fairly bad regardless of whether I'm 
writing packets to a file or to RAM - I tend to lose at least 20% even 
at reasonably low rates (10MB/s),

The problem that's worrying me at the moment is the ring buffer issue - 
I suppose my disk speed limits may be making the problem worse, but the 
drop rate seems unreasonably high considering the load.  It's similar to 
a problem I had with ring buffer in Windows - I found that while I could 
write files in ring buffer at rates of up to about 45MB/s (and 
potentially more - I haven't tested higher rates) with drop rates of 
0.005%-0.1%, but once I started wrapping files, drop rates jumped to 
about 50%.

Any thoughts here?  I have a few things to change/check out, and I'd 
like to review the ring buffer code for insights, but am I the only one 
having such terrible drop rates with the RB mode in Linux?

Thanks again,
Ian

Guy Harris wrote:

> On Thu, Mar 13, 2003 at 04:49:50AM -0500, Ian Schorr wrote:
>  
>
> > Tethereal is reporting only seeing a certain number of frames (again, 
> > 30-70% or so), and reports 0 dropped.  I'm not 100% sure whether the 
> > kernel is compiled to allow drop reporting, so I'm not sure what this 
> > symptom is telling me.
> >    
>
> It's probably more of an issue of whether *libpcap* is compiled to allow
> drop reporting.  I think all 2.4[.x] kernels that support packet
> capturing at all should allow userland code to get the statistics;
> however, libpcap prior to 0.7 didn't include code to get them, and even
> 0.7 and later releases will support it only if compiled on a platform
> that supports it - which might not be the case if compiled on, for
> example, a platform with 2.2[.x] header files.
>
>  
>
> > Any suggestions on what might be happening here?
> >    
>
> It might be interesting to see whether this is due to packets being
> dropped because there's not enough free space in the socket buffer; if
> RH 7.3 doesn't have libpcap 0.7 or later, try downloading the current
> CVS version of libpcap, building and installing it (make sure that the
> "config.h" file in the top-level libpcap source directory defines
> HAVE_TPACKET_STATS; if not, you're on a platform with old header files),
> and then get the Ethereal source if you don't already have it,
> reconfigure it to pick up the version of libpcap you installed rather
> than the system libpcap (probably using "--with-pcap=/usr/local", as
> tcpdump.org's libpcap is installed under "/usr/local" by default), and
> rebuild it (when you're done, run "ldd" on the Ethereal and Tethereal
> binaries to make sure they're *NOT* dynamically linked with libpcap). 
> Then see whether they report any packets being dropped.
>
> _______________________________________________
> Ethereal-users mailing list
> Ethereal-users@xxxxxxxxxxxx
> http://www.ethereal.com/mailman/listinfo/ethereal-users
>
>