[John Dill <John.Dill@xxxxxxxxxxxxxxxxx>]

>Message: 1
>Date: Fri, 20 Nov 2020 17:43:33 -0800
>From: Guy Harris <gharris@xxxxxxxxx>
>To: Developer support list for Wireshark <wireshark-dev@xxxxxxxxxxxxx>
>Subject: Re: [Wireshark-dev] wireshark capture/filtering question
>Message-ID: <31E12E1E-224B-4223-AF81-659E1B6BF258@xxxxxxxxx>
>Content-Type: text/plain;       charset=us-ascii
>
>On Nov 20, 2020, at 11:02 AM, John Dill <John.Dill@xxxxxxxxxxxxxxxxx> wrote:
>
>> Not exactly.  What I'm looking to do is to merge our existing 1553 capture
>> C code and wireshark capture code (inspired from tshark or dumpcap) into
>> the same application.
>>
>> The 1553 data part would get passed records as is over a TCP socket to a
>> dashboard application for display (not injected into Wireshark).  This
>> application interfaces with a PCMCIA card and the 1553 data is stored
>> in a queue of fixed length records.  Those records are then read and
>> streamed by a C program using TCP packets to a dashboard application
>> that reads them, decodes all the data fields and puts them on a GUI for
>> display.
>
>OK, so what programs are involved here?
>
>Is the PCMCIA card something that plugs into the (presumably) MIL-STD-1553
>serial bus and captures traffic from it?

First off, I appreciate the time spent in your response.  Sorry, I don't know
how far into the weeds to go with this, but I'll do my best.  Thanks for your
patience in advance.

The MIL-STD-1553 interface is an avionics 10 MHz serial data bus that has its own
message interface.  All the equipment is connected via BNC connectors to bus
terminals.  There is a central application called the Bus Controller that schedules
messages between various avionics devices.  The serial bus is shared between
all avionics, so each message is seen by all the devices, but unless the message
remote terminal (RT) address matches the avionics device, the message is ignored.

The PCMCIA card is able to read the 10 MHz waveform and save these messages
into the local vendor's "frame" format.  Most vendors provide a standard bus
monitor program that reads the message data from the PCMCIA card, but they
don't have a mechanism to translate the data into engineering units for display.
They have an application library that takes care of the low level device driver issues
to read 1553 packets into their vendor "frame" format.  They typically provide
several proof of concept tasks that you can perform with their 1553 card using
their API, but nothing complex enough to be really useful.

>If so, is that what your existing 1553 capture C code reads from?

The 1553 capture C code reads from a Excalibur Systems EXC-1553 PCMCIA/EP
card.  It is a custom written application that reads 1553 messages in the vendor
"frame" format using the vendor provided 1553 library, and has the capability
to write the messages to disk, or to send the 1553 messages as packets over TCP
to a Java based dashboard application for display.

So far, all of this code is written in house (and was not written by me either).

>What program sends those records on a TCP socket?  Is that the existing C code?

The existing in house capture application sends 1553 messages over a custom
protocol using the TCP as the connection layer.  It doesn't send TCP data until a
client application establishes a connection.  After the dashboard initiates
a socket connection, there is a simple control protocol to the 1553 capture
application to start and stop sending 1553 traffic to the dashboard application.

>What program reads from the TCP socket?  The dashboard application?

The dashboard (Java) application reads 1553 packet frames sent from the capture
application (C).

>And are you describing the code that exists *now*, or the code you're talking
>about that, in some fashion, includes some Wireshark code?

There is no Wireshark code at this point, it's all completely in-house built software.
This is how the code functions now.  As an aside, the PCMCIA operates in a Bus
Monitor mode, which means it's not supposed to interact in any way with the
existing avionics devices on the 1553 bus.

>> This application is also capable of writing captured 1553 data
>> in fixed length records to a file.  The application can also load a 1553 data
>> capture file and streaming the records to the dashboard application to
>> simulate a playback of a flight at varying playback speeds.
>
>> The packet part of the data would be captured from a mirror port and
>> get filtered by the bpf capabilities to eliminate packets of non-interest,
>
>BPF doesn't know anything about MIL-STD-1553.  You could write BPF filter
>language expressions that test fields at particular offsets (or
>generate your own BPF machine code), but that would only require a small
>part of libpcap's capability - it wouldn't require any Wireshark code at all.
>
>Or do you mean using BPF to filter TCP packets arriving over, for example, Ethernet?

At the core of this avionics platform is an ethernet switched network.  The
main operational flight program has its own internal 1553 interface and acts
as the Bus Controller.  There are multiple applications on several display devices
that have their own VMs and use ethernet via UDP and TCP connections for
command and control and to receive data.  I'm able to monitor this traffic using
port mirroring, and have custom dissector plugins to display the packet contents
in Wireshark.  One example of this is that I'm able to visualize all the key presses
and display screens through custom dissector given packets on this mirror port.

So far, this is completely separate from the 1553 data capture and visualization
application described previously.  I use Wireshark to visualize the key presses
and displays, which is ethernet only, no 1553 data is involved in the displays.

>If your existing program is sending the packets over a TCP socket, and you
>want to use libpcap/WinPcap/Npcap to capture that traffic and extract it,
>is there some reason you want to do that rather than just listening on the TCP socket?

Currently, I'm relying on Wireshark packet desegmentation and packet filtering
capabilities to isolate the key presses and 24x15 character displays.  If I just
listen on the port mirrored data, I'd be overwhelmed with packet traffic and
I'd need to duplicate the capability that bpf filter feature and TCP desegmentation
that Wireshark provides.

>Currently, some remote capture protocols are handled in Wireshark by sniffing
>the network for them and dissecting them within Wireshark, with the dissector
>proceeding to hand the captured packets within the capture protocol packet
>to the appropriate dissector, but that's a *HORRIBLY* greasy hack, which would
>be much better handled by libpcap being able to just listen for that traffic using
>the regular OS networking stack and hand them to an application (tcpdump,
>Wireshark, whatever).  That's on my to-do list; step 1 is making rpcap capable
>of using the newer pcap_create()/pcap_activate() APIs, and then step 2 is to
>add support for *other* URL schemes for use with other protocols.
>
>> and then get passed in some manner to the same dashboard application over TCP.
>> I have a NDO dissector that can heuristically detect those packets of interest,
>> like "ndo.id == CDU_DM_Display_Buffer" or "ndo.id == CDU_DM_Key", so I'd
>> like to use that capability to detect when these payloads of certain packets
>> are found.
>>
>> This includes the TCP desegmentation capabilities as well,
>
>OK, so you need Wireshark's full dissection capabilities, meaning dumpcap can't do it alone.

OK, that is good to know.  Sounds like tshark is the starting point for heavy study.

>> since the CDU
>> Display packet is segmented over 3 packets.  Once a complete packet with
>> the NDOs of interest is detected, the dashboard would get a copy over a
>> separate TCP stream.  The NDOs would be packaged into new frames and
>> sent via TCP to the dashboard application for display.
>>
>> The capture application also needs to take these NDOs of interest when
>> they are detected, generate frames that can be multiplexed with 1553
>> data to be saved to file in the chapter 10 format.  chapter 10 format is a
>> IRIG 106 avionics standard for multiplexing different data modalities,
>> which I'm planning to use to have a single file to playback the flight.
>>
>> As a starting point, I'm looking to see if there's a way that I can read
>> a packet stream, detect NDOs of interests, but instead of writing
>> dissected information to the screen or a file, I want the output to
>> be over a TCP socket that's established from the dashboard application.
>
>So the goal here is to add filtering capability to the existing "read 1553
>packets and send them over a TCP socket to the display application" data path?

Nope.  I'm not looking to filter any of the 1553 traffic.  What I'm looking to
do is to take the capability of tshark to do bpf filtering to filter out all the
key and display data from the port mirrored avionics ethernet stream, and
pipe it into the dashboard application so that I can get basically a virtual
display.  Eventually I would amalgamize the C programs of the 1553 portion
and the bpf filtered port mirrored display data together.  This would use
the chapter 10 format from IRIG 106 as the file format, which is outside
the scope of Wireshark.  That problem is future me's problem right now.

>From an conceptual perspective, what I'm looking for is a way to inject
a custom code pathway so that when I get a successful packet dissection
of a CDU_DM_Key NDO or a CDU_DM_Display_Buffer NDO (the 2nd
requires TCP packet desegmentation), I can have a callback to reframe
this data and then pipe the new packets to the dashboard application
over a custom TCP socket connection (independently of what's already
happening with the 1553 data capture).

This could include re-framing the NDO message to strip out all the other
NDOs not matching those two above, since packets can have multiple
NDOs with different NDO IDs, but I only want the output of this stream to
be just 2 NDO IDs.  It also could include comparing the display PDU
against the previous and only sending it if the display has changed in
any way to reduce the amount of data sent (which would reduce the size
of the capture file).

Essentially, I want to use tshark as a kind of passthrough that takes the
hundreds of NDOs in the port mirror network stream and filters it down
to 2, re-framing PDUs into custom packets as needed.  Eventually this
will need to be combined with the 1553 data capture to have a unified
timestamp for both 1553 data and ethernet display data.

There are two approaches to this.  One is to extract all the components
necessary from tshark and incorporate it into the 1553 data capture
application.  The other is to inject the components of the 1553 data
capture application into a custom tshark.

>Note that Wireshark isn't required to use libpcap for all its capturing;
>there's a mechanism called "extcap":
>
>        https://www.wireshark.org/docs/wsdg_html_chunked/ChCaptureExtcap.html
>
>which allows a program to get packets from some source *other* than
>libpcap/WinPcap/Npcap and feed them to Wireshark's packet capture code.
>
>That *would* require that you get a pcap/pcapng link-layer header type for
>1553, but it would mean that you could directly feed a 1553 packet stream
>to Wireshark/TShark.

I'll do some more reading up on that.

Just to re-iterate, I'm not intending to feed the 1553 data captured from the
Excalibur EXC-1553 PCMCIA/EP card into Wireshark.  Technically, the port mirrored
stream already has a feed of the 1553 data encapsulated into NDO packets.
I already have a dissector for it (but it's just generic it doesn't translate the
1553 messages to display engineering units since we already have the dashboard
application for that).

Mostly I want to figure out how to trigger my own custom codepath when I get
a successful dissection of two NDO packets using bpf filter expressions like
"ndo.id == CDU_DM_Display_Buffer" or "ndo.id == CDU_DM_Key".  Then I'd like
to determine whether it's simpler to hack a custom tshark to accomplish it, or if
I can use the components and graft it into the existing 1553 data capture application.

Sorry, the message is a bit rambly.

Best regards,
John D.