[Jon Polacheck <jonpol@xxxxxxxxxxxxx>]

I am working on a continuous packet capture application (think
Infinistream, Gigastor, NetVCR) written in Perl (only because that's
what I know). Here is what I have so far.


This works (or seems to).  More optimization possible by removing
unnecessary trailing spaces from each packet hexdump.  Hacked
Data::Hexdumper some, more probably doable.

Used mkfifo to create the named pipe.  Perl sees it as a disk file
(that I called qtfifo).  ENQUE.pl dumps packet hexdumps to the fifo. 
DEQUE.pl reads lines from the fifo.  /^0000 / acts as the delimiter. 
mysql compression worked with the standard OpenSuSE install, no 
recompiling or other mucking about necessary.

Lines used for debugging marked as such.

ENQUE.pl

use Net::Pcap;
use Data::Hexdumper qw(hexdump);

$dev = "eth0";

# used a 50 packet cap file to make sure what came out matched what went in
#$dump = "ip.cap";
#$pcap = Net::Pcap::open_offline($dump, \$err) or die "Can't read
'$dump': $err\n";

# live, real-time feed
$pcap = Net::Pcap::open_live($dev, 1514, 1, 0, \$err);

Net::Pcap::loop($pcap, -1, \&process_pkt, ""); # <- subroutine call

sub process_pkt {
   open(QT, "> qtfifo");
# $_[2] is the third element of the default array "@_" which was created
# by the subroutine call "&process_pkt"
   my $pkt=$_[2];
   $results = hexdump( data => $pkt
       , number_format => 'C',
   );
   print QT $results;
   close(QT);
    $i++; # debug
    &stop_run if $i > 100; #debug
}

# all debug below
sub stop_run {
 print "stop_run\n";
 open(QT, "> qtfifo");
 print QT "\nx\n";
 close(QT);
 print "enque ended\n";
 exit;
}


DEQUE.pl

use Time::HiRes ( nanosleep );
use DBI;

$hostname="127.0.0.1";
$database="cpc";
$port="3306";

$dsn = "DBI:mysql:database=$database;host=$hostname;port=$port";

$dbh = DBI->connect($dsn,
   "root",
   "",
   {'RaiseError' => 1});

# call the Net::Packet collector script
system(q{perl ENQUE.pl&});

open(EQT, " < qtfifo");

$i = 0; # debug
$pc = 0; # debug

while(1) {
   $i++; # debug
   $line = readline(EQT);
   if ($line =~ /^0000 / ) {
       $dbh->do(qq{INSERT INTO cpc VALUES ( compress("$pkt"))});
       $pc++ if defined($pkt); # debug
       print "packet $pc:\n$pkt\n" if defined($pkt); # debug
       undef($pkt);
       $pkt .= $line;
   } else {
       $pkt .= $line;
       &theend if $pkt =~ /x/; # debug
   }
   nanosleep(1); # would not work without this!
}

# all debug below
sub theend {
   close(EQT);
   print "$i loops\ndeque ended\n";
   exit;
}

This generated a cap file that looks just fine in Wireshark.

mysql -Br -D cpc -e "select uncompress(packet) from cpc;" | text2pcap -
m_cap.cap

Hope you find this of interest.

Jon Polacheck