Question about vt6: List for an unknown topic - 0xFF cursor

Challenges
1

0xFF can’t be how you indicate a null cursor, right? It would lead to completely ambiguous parsing that you could only figure out by attempting to parse the entire message two different ways, unless there’s also an undocumented byte that indicates a non-null cursor object. My reasoning is as follows:

The first property of the cursor is a COMPACT_STRING. I can accept this field being nullable (aka COMPACT_NULLABLE_STRING rather than COMPACT_STRING) since they’re the same spec, except that COMPACT_STRING doesn’t define the null case (meaning a length field of 0 is undefined). However, even as a COMPACT_NULLABLE_STRING, 0xFF still wouldn’t be a null value - I’d expect 0x00 to indicate null. That’s the whole point of the N+1 length encoding - 0 is null, while 1 is a field of length 0 (i.e. null vs empty).

If 0xFF did mean a null cursor, how would you represent a string of length 255? It should be 0xFF 0x01, but that would be interpreted as a null cursor followed by an empty but non-null tagged fields list. The only way to figure out which one it meant, baring an undocumented byte being included before the string, is to try both ways and see which one results in a valid interpretation of the request.

Sources:

COMPACT_NULLABLE_STRING
Represents a sequence of characters. First the length N + 1 is given as an UNSIGNED_VARINT . Then N bytes follow which are the UTF-8 encoding of the character sequence. A null string is represented with a length of 0.

Compact String
A compact string field contains a 32-bit unsigned varint, followed by the string bytes.
If the length field is 0, the string field is null. If the length field is 1, the length is 0. If the length field is 2, the length is 1, etc.

Unsigned Varints
The UNSIGNED_VARINT type describes an unsigned variable length integer.
To serialize a number as a variable-length integer, you break it up into groups of 7 bits. The lowest 7 bits is written out first, followed by the second-lowest, and so on. Each time a group of 7 bits is written out, the high bit (bit 8) is cleared if this group is the last one, and set if it is not.

3

Aha, there is indeed documentation for it - just not on the protocol docs:

So, -1 if null, 1 if not null. That means a request with a cursor will be 0x01 <UNSIGNED_VARINT length> (etc...)

The fact that the nullability of the struct is not specified seems to be a problem with the documentation generation, then.

I’d suggest calling out the nullability explicitly, since it isn’t implemented on docs. Even if there aren’t any plans to set the cursor in future stages, saying something like: “The cursor field is nullable, which means it will 0xFF if null, or 0x01 followed by the documented fields if not null. For this challenge, it will always be null.” That would help a lot in avoiding confusion when just trying to parse everything and getting an error because of an unexpectedly large string length.

Noting any other nullable structs, if there are any in later stages, would be good too.

2 Likes
4

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.