Build your own Kafka (Go edition)
I’m not stuck per se - I can work around the seemingly invalid request - but:
I don’t believe the request that is being sent for testing stage ‘List for an unknown topic’
(The Software Pro's Best Kept Secret.) is correct.
Several bytes are missing at the end, if it is supposed to follow the protocol
specification.
DescribeTopicPartitions Request (Version: 0) => [topics] response_partition_limit cursor TAG_BUFFER
topics => name TAG_BUFFER
name => COMPACT_STRING
response_partition_limit => INT32
cursor => topic_name partition_index TAG_BUFFER
topic_name => COMPACT_STRING
partition_index => INT32
Following the hexdump of the request payload below
- After the ‘unknown topic’ string, there’s an empty TAG_BUFFER (1 byte).
- Then 4 bytes goes towards
response_partition_limit
(0x00000001
). Then
there’s only 2 bytes left (0xFF00
). 0xFF
on its own is not valid as an UNSIGNED_VARINT (which is what COMPACT_STRING)
starts with (looks like a signed int8? - meant to signal a nullable string?)
Either way, that’s not to mention the 4 bytes from partition_index
and 2 tag_buffers
(one for cursor and for the request as a whole)
(Note: the dump does not including the leading 4 bytes for request length)
0: 0000002d 004b0000 514df2b3 000c6b61 | ░░░-░K░░QM░░░░ka
16: 666b612d 74657374 65720002 0e756e6b | fka-tester░░░unk
32: 6e6f776e 2d746f70 69630000 000001ff | nown-topic░░░░░░
48: 00 ^ ^ | ░
| |
| + end of response_partition_limit
+-end of 'unknown topic' string
After response_partition_limit, there would have to be atleast 7 bytes:
- 1 byte for empty compact string
- 4 bytes for partition index
- 1 byte for tag_buffer (cursor)
- 1 byte for tag_buffer (request)