Wireshark-users: Re: [Wireshark-users] extraction of files from SSL and TCP streams automatically

From: Miroslav Rovis <miro.rovis@xxxxxxxxxxxxxxxxx>
Date: Tue, 8 May 2018 08:45:55 +0000
On 180507-13:40+0200, Peter Wu wrote:
> Hi Miroslav,

Hi Peter, nice to read from you again!
( firt time it was in the thread I started at:
Filtering on (negated) frame.time_relative filters out wrong frame.number
https://www.wireshark.org/lists/wireshark-users/201703/msg00030.html
where I corresponded with you and with the kind programmer Graham Bloice :-) .
(And that was a real, but silent, bug there, I know because later Wiresharks
took the same commands and did the right output, and so did editcap, while the
then version of Wireshark, including edicatp, did not. But that's past now.)
 
> On Sat, May 05, 2018 at 06:17:42PM +0000, Miroslav Rovis wrote:
> > Hi!
> > 
> > How do users climbing the steep path of deep packet inspection extract files,
> > in HTTP/HTTPS protocols, i.e. the streams in SSL (and plain TCP) conversations?
> > 
> > Is there a program that can extract files from SSL- or plain- TCP streams
> > automatically?
Hmmh... Of course, in Wireshark it's not done automatically... But maybe, and
if you are saying so then yes, in Tshark it must be that it can be done
automatically...

Can I ask when did Wireshark/Tshark get that ability? Already years ago or
relatively recently...

Just for the sake of history.

I remember I later read how, I think it happened thanks to the Wireshark
developer Sake Blok, somewhere in 2014 --or was it in 2012?--, only then,
Wireshark, and it must have been Wireshark were the first team in the world to
accomplish that... became capable of extracting streams from SSL-encrypted
conversations... Before that time, it just wasn't possible to decrypt SSL...
That much of history I do know.

So when did Wireshark/Tshark get the ability to extract objects from streams?

> [..]
> > And I've managed to put together a script that uses a few modified
> > subroutines from Chaosreader on already decrypted SSL TCP streams and extracts
> > files from them.
> 
> I think the feature you are looking for is "Export HTTP Objects". In the
> GUI this is accessible via File -> Export Objects -> HTTP.
> 
> Since Wireshark 2.4, this feature is also available in tshark. For
> example, to save all files from HTTP bodies in directory "outputdir":
> 
>     tshak -r some.pcap --export-object http,outputdir
> 
> See also https://www.wireshark.org/docs/man-pages/tshark.html
Hmmmh... I think I see what you mean.

> Hope it helps.
And probably Tshark can do as good as my stream-cont.pl? Extract all files even
better maybe...

Thinking loud now... Actually postponing some more thoughts/work (see below,
"other obligations", and "quick reply"):

So what would be the commands to issue, then, on the trace that I offered, and
which my stream-cont.pl on streams produced from that trace with my
tshark-streams.sh, extracted all the files out from, as I show on that
explanation page of mine at:

https://www.croatiafidelis.hr/foss/cap/cap-180505-schmoog-referendum/

It's not that I'm lazy or disrespectful, but I simply do not have not even one
half hour now to try and figure out the right commands, because of other
obligations. This can only be a quick reply.

Anybody else reading here, and knowing the commands to use to do the "tshark -r
PCAP --export-object"'ing, thanks if you jump in.

And so if that does it just like my script does it (or better), then yes, that
helps and thank you, Peter!

-- 
Miroslav Rovis
Zagreb, Croatia
https://www.CroatiaFidelis.hr

Attachment: signature.asc
Description: PGP signature