When building the Kafka challenge, we struggled quite a bit to understand Kafka’s protocol format and on-disk storage format. The docs are very confusing (non-existent in some cases), and we had to resort to reverse-engineering Kafka’s source code.
This got us thinking… this is something we struggle with in almost every challenge. Redis RDB files, Git object files, SQLite database files, Bittorrent peer messages - all of these are non-trivial binary formats, just that some are better documented than the others.
We built a tool to make learning these formats easier! We’re calling it BinSpec, here’s how it looks:
It’s live now and you can try it out: binspec.org (works well on desktop, not so great on mobile).
Please let us know if you’ve got any feedback on this, and if you’d like us to add formats for other challenges!