Hi all,
I might be missing something, so please let me know. At the moment I am trying to parse the cluster metadata file with several batches inside. I noticed however that the record batch one has a length parameter which is a single byte, and in record batch 2 the length is a double byte. I am struggeling a bit with which one is correct, or if I should be able to detect this somehowe from the message (or length) itself. Can somebody give me a pointer on this?
Thank you!
Wouter van Veen
Hey @woutervanveen, could you use BinspecVisualizer and share a screenshot of the specific fields you’re referring to? Thanks!
That was quick!
This is the length of record 1:
I wanted to put a screenshot of the second batch second record, but I am limited to a single screenshot, so I will put it in the next post
This is the screen shot of the second record of the second batch:
Actually the length of the first record in the second batch also uses only 1 byte.
@woutervanveen Ah, gotcha. You’ll want to look into how varint encoding works.
Here’re two related posts:
Let me know if it still does not make sense after that.
Here’s a quick verification of 0x9001 as 72:
0x9001 has two bytes:
1001 0000: continue0000 0001: stop001 0000 + 000 0001 reverses order due to endianness:
000 0001 + 001 0000
= 1 001 0000 = 144
144 becomes 144 / 2 = 72 due to zig-zag encoding.
Thank you! That should be enough information to fix this!
This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.