This is for Stage #EA7.
I’ve been assuming that the any length is 1 byte, but then I saw that the value length of the record is 2 bytes. After further reading, I realized that the spec said its a signed variable size integer. How does one parse a variable sized integer i.e how does one know when to stop reading a variable sized thing?
Hi @pdtran3k6, here’s an example illustrating how to parse 0x8147
as decimal 199.
how does one know when to stop reading a variable sized thing
Stop when a byte has a leading 0.
0x8201 has two bytes:
1
000 0010: continue
0
000 0001: stop
what’s the software that you use here to see the hex in binary like that?
and also, if I read the other post correctly, in this case, the final number after concat would be:
000 0010 + 000 0001
= 10 000 0001
= 1 0000 0001 = 257
which is definitely not 65 like the link here
and even if I intepret it as a signed integer, that would be -255, also not 65
It’s the Calculator app on mac in programmer mode:
You’re absolutely right! It’s 257. I was a bit confused as well, because the length of Value (Partition Record)
is apparently 65 (= 1 + 16 x 4):
Turns out that VARINT in Kafka involves two additional considerations:
- Small/Big-endian order
- Zig-zag encoding
000 0010 + 000 0001 reverses order due to endianness:
000 0001 + 000 0010
= 1 000 0010 = 130
130 becomes 130 / 2 = 65 due to zig-zag encoding.