Hi,
Attached please find an incomplete, work-in-progress improved dissection of the VNC protocol. While it's completely not ready, it's a bit better than the current one, I believe.
Specifically:
1. Added a new kind of pseudo encoding (
http://www.tigervnc.com/cgi-bin/rfbproto#extendeddesktopsize-pseudo-encoding) - will submit separately as a patch for current dissector.
2. Complete overhaul of the VNC desegmentation code (which did not really work). The way I'm doing it may be wrong, but: for each message X I have a walk_message_x() function which parses it just enough to get the full length needed to actually parse it. First of all, it is inefficient - if I don't have enough bytes, I ask for more (as much as I know at that point that I'll need - regretfully, with VNC, there are no shortcuts) - and re-parse it from start when I get the additional bytes. I don't have a better solution to it right now. I don't think it's worth the complexity to 'store' the knowledge I know so far of the message, and continue from there. Secondly, it almost doubles the code - one to parse, and very similar to actually do the tree and items stuff.
3. Corrected hextile encoding parsing. It's quite wrong the way it's done today (see 2nd rectangle in packet 23 of the attached sample capture). It completely ignored the fact that the hextile encoding is actually encoding a series of 16 by 16 pixel tiles (hence hexTILE, I reckon). This presents another problem which I don't know how to solve - I'm getting too many (legitimate!) items in my tree - and wireshark thinks I'm in an infinite loop. The fact it, each update may have several rectangles, each may have subrectables, each may have tiles... it's endless if the screen is big enough.
Any ideas how to solve the two issues, or if anyone wants to carry the work - I'd appreciate that.
TIA,
Y.
Attachment:
hex3.pcap
Description: Binary data
Attachment:
packet-vnc.c-dseg.diff
Description: Binary data