# Question about Handle APIVersions requests stage **URL:** \ **Category:** Challenges\ **Tags:** challenge:kafka\ **Created:** [September 14, 2024, 11:17pm UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743 "2024-09-14T23:17:50Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![rohitpaulk](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/rohitpaulk/32/6_2.png) [@rohitpaulk](https://forum.codecrafters.io/u/rohitpaulk)\ **Post date:** [September 14, 2024, 11:17pm UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/1 "2024-09-14T23:17:50Z")
@Eghizio moving this here. > The instructions seems to be off with what the actually is being tested: > > - The first 4 bytes of your response (the “message length”) are valid. > - The correlation ID in the response header matches the correlation ID in the request header. > - The error code in the response body is `0` (No Error). > - The response body contains at least one entry for the API key `18` (API\_VERSIONS). > - The `MaxVersion` for the `ApiKey` `18` is at least `4`. > > While the tester seems to check more than that (for example `throttle_time_ms` which is not even mentioned in any of the challenge steps): ```auto remote: [tester::#PV1] [Decoder] - .ResponseHeader remote: [tester::#PV1] [Decoder] ✔️ .correlation_id (810919229) remote: [tester::#PV1] [Decoder] - .ResponseBody remote: [tester::#PV1] [Decoder] ✔️ .error_code (0) remote: [tester::#PV1] [Decoder] ✔️ .num_api_keys (0) remote: [tester::#PV1] [Decoder] ✔️ .throttle_time_ms (256) remote: [tester::#PV1] Received: remote: [tester::#PV1] Hex (bytes 21-25) | ASCII remote: [tester::#PV1] ------------------------------------------------+------------------ remote: [tester::#PV1] 00 00 00 00 00 | ..... remote: [tester::#PV1] ^ ^ remote: [tester::#PV1] Error: Unexpected end of data remote: [tester::#PV1] Context: remote: [tester::#PV1] - ApiVersions v3 remote: [tester::#PV1] - Response Body remote: [tester::#PV1] - TAG_BUFFER remote: [tester::#PV1] - TAGGED_FIELD_ARRAY remote: [tester::#PV1] - UNSIGNED_VARINT remote: [tester::#PV1] remote: [tester::#PV1] Test failed ``` Also there is a mention of `ApiVersions v3` where in the challenge it is written that the tester `will use v4`. ---
**Author:** ![rohitpaulk](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/rohitpaulk/32/6_2.png) [@rohitpaulk](https://forum.codecrafters.io/u/rohitpaulk)\ **Post date:** [September 14, 2024, 11:19pm UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/2 "2024-09-14T23:19:08Z")
We’ll update the docs soon to mention v4 - this was a last minute change to match a soon-to-be-released version of Kafka that introduces v4. Details on this here: [kafka/clients/src/main/resources/common/message/ApiVersionsRequest.json at 7a321f29a2495d519165206e50b10595a0ceae9c · apache/kafka · GitHub](https://github.com/apache/kafka/blob/7a321f29a2495d519165206e50b10595a0ceae9c/clients/src/main/resources/common/message/ApiVersionsRequest.json#L25-L26) ---
**Author:** ![rohitpaulk](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/rohitpaulk/32/6_2.png) [@rohitpaulk](https://forum.codecrafters.io/u/rohitpaulk)\ **Post date:** [September 14, 2024, 11:20pm UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/3 "2024-09-14T23:20:50Z")
Re: throttle\_time\_ms, we’ll add this to that list! Screenshot from official docs: ![Screenshot 2024-09-15 at 00.19.25](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/2X/8/83095810783aa09b7edfb6a41b1017a035905cf4.png) ---
**Author:** ![romainfd](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/romainfd/32/1681_2.png) [@romainfd](https://forum.codecrafters.io/u/romainfd)\ **Post date:** [September 15, 2024, 9:39am UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/4 "2024-09-15T09:39:38Z")
Hello ! Since we don’t parse the Request content, I am curious why not only implement ApiVersions Request v2 ? Best, Romain ---
**Author:** ![ValentinJub](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/valentinjub/32/145_2.png) [@ValentinJub](https://forum.codecrafters.io/u/ValentinJub)\ **Post date:** [September 15, 2024, 10:31am UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/5 "2024-09-15T10:31:27Z")
Since I spent the last two hours trying to determine what was expected from the tests and it wasn’t fun because the instructions are unclear and nowhere can we find the right sequence that’s expected. I’ll share them here since it doesn’t seem the format of the expected response is listed on Kafka doc, it’s not exactly the v3 response as @rohitpaulk shared but rather something a bit different: ApiVersions Response (Version: CodeCrafters) =\> error\_code num\_of\_api\_keys [api\_keys] throttle\_time\_ms TAG\_BUFFER error\_code =\> INT16 num\_of\_api\_keys =\> INT8 api\_keys =\> api\_key min\_version max\_version api\_key =\> INT16 min\_version =\> INT16 max\_version =\> INT16 \_tagged\_fields throttle\_time\_ms =\> INT32 \_tagged\_fields > **Further explanations** > > For the num\_of\_api\_keys I had to input 2 despite having only one API key, not sure why but the decoder seems to decode my 2 as a 1, when I input 1 it errors in the decode. > > The \_tagged\_field is just a single byte with a value of 0 ---
**Author:** ![romainfd](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/romainfd/32/1681_2.png) [@romainfd](https://forum.codecrafters.io/u/romainfd)\ **Post date:** [September 15, 2024, 11:22am UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/6 "2024-09-15T11:22:46Z")
For num\_of\_api\_keys, I think this is just the way lengths are encoded (see COMPACT\_ARRAY type definition) : > Represents a sequence of objects of a given type T. Type T can be either a primitive type (e.g. STRING) or a structure. First, the length N + 1 is given as an UNSIGNED\_VARINT. Then N instances of type T follow. A null array is represented with a length of 0. In protocol documentation an array of T instances is referred to as [T]. I don’t understand why they do this instead of just using 0 for 0, 1 for 1, … but adding this extra + 1 for non empty arrays made it work Regarding the TAG\_BUFFER, they said we don’t use tag in the CodeCrafter challenge, so it os just one byte to encode a length of 0 for the tag array. At least, that’s the more or less logical explanation I came up with after being puzzled just like you are and deriving this 0 on one byte empirically [TAG\_BUFFER explanation · romainfd/codecrafters-kafka-python@8b3ecf9 · GitHub](https://github.com/romainfd/codecrafters-kafka-python/commit/8b3ecf9aa78df0a5098d0463d46e997f6581aac7) So I agree the instructions are not super clear but I think this is also a problem of the Kafka doc not always being super clear… I also had a LOT of trouble finding out when to use Response Header V0 vs V1… ---
**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\ **Post date:** [September 18, 2024, 4:32pm UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/8 "2024-09-18T16:32:26Z")
> Since we don’t parse the Request content, I am curious why not only implement ApiVersions Request v2 ? @romainfd We implemented `ApiVersionsRequest:v4` to stay on the bleeding edge of Kafka. Our users probably don’t want to implement older versions of requests. * * * > I don’t understand why they do this instead of just using 0 for 0, 1 for 1, … but adding this extra + 1 for non empty arrays made it work For `COMPACT_ARRAY`, the reason why Kafka uses N + 1 as the length is to differentiate a **null array** (represented as 0) and an **empty array** (actual length of 0, but represented as 1). * * * > I also had a LOT of trouble finding out when to use Response Header V0 vs V1 If you don’t need to support older versions of Kafka, use Response Header **v1** , because of [KIP-482](https://cwiki.apache.org/confluence/display/KAFKA/KIP-482%3A+The+Kafka+Protocol+should+Support+Optional+Tagged+Fields): > In order to have flexible version support across all requests and responses, we will bump the version of all requests and responses. > … > The new ResponseHeader version will be version 1, superseding version 0. The only exception is that “`ApiVersionsResponse` always includes a **v0** header”. > ---
**Author:** ![CarlosLanderas](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/carloslanderas/32/1819_2.png) [@CarlosLanderas](https://forum.codecrafters.io/u/CarlosLanderas)\ **Post date:** [September 21, 2024, 9:28am UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/9 "2024-09-21T09:28:50Z")
Reverse engineering master. Instructions should be improved A LOT ---
**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\ **Post date:** [October 4, 2024, 1:22am UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/10 "2024-10-04T01:22:29Z")
---
**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\ **Post date:** [October 4, 2024, 1:22am UTC](https://forum.codecrafters.io/t/question-about-handle-apiversions-requests-stage/1743/11 "2024-10-04T01:22:37Z")
Closing this thread due to inactivity. If you still need assistance, feel free to reopen or start a new discussion!