Howdy,
Sorry for the long email, but it's better than wasting time coding something people won't accept. :)
I have a need to add yet another file format for Wireshark to read from (not write/dump to), which I've done before in C. It occurs to me, though, that my use-case this time would be better served by writing a Lua plugin to do it instead. The reason is this is for a vendor-proprietary file format that changes now and then in subtle ways, so it has to be updated/fixed to handle new ones, and preferably quickly (ie, without waiting for the next wireshark release). The number of wireshark users for these formats would be small (like <5k), and it's for text-based files, so I feel a Lua plugin model for a file format reader makes sense. Obviously the current Lua plugin model doesn't handle that, but I'm willing to make the changes and submit the diffs. I've written Lua bindings before for production code, and I can follow the model wireshark currently uses.
To do this, I'm planning to let plugins register a Lua func to be pcall'ed during the wtap_open_offline() walking of the open_routines. The registration would let the Lua plugin decide to be the first one tried, last one, or in the middle (ie, be the first heuristic one). When the registered Lua func gets called, the Lua plugin would need to decide if the file type belongs to it or not, just like C-based ones do; and it needs to set the wtap struct members like C-based readers do, etc. The Lua func can then attach more Lua functions to be called for the read/seek/close functions.
In other words the basic idea is to follow the existing wtap file open/read/seek/close model done by C-based readers, but invoke registered/attached Lua functions for them instead; and of course expose in Lua the necessary functions to set the wtap struct's members, create buffers, etc.
So... the questions:
1) If I do this, what are the chances it gets accepted and delivered in a future wireshark release? Are there any general concerns with allowing Lua plugins to handle wtap file reading, in principle?
2) Has anyone else already done this or is in the process of doing this?
I have some implementation questions as well - there are some choices to make for how to do this - but I'll email them if the response isn't too negative to the general idea.
-hadriel