Some tests require response header in v0 (ApiVersions) other in v1 (Fetch).
When I updated to v1 - e.g. added empty TAG BUFFER it fails with ApiVersions, but is required for Fetch
Why? Found nothing in Kafka protocol docs, that would explain it
Log:
your_program] 2024-10-12T20:04:14.810041Z DEBUG kafka_starter_rust: accepted new connection peer_addr="127.0.0.1:39744"
[your_program] 2024-10-12T20:04:14.810525Z DEBUG kafka_starter_rust: Got frame frame_length=48
[your_program] 2024-10-12T20:04:14.810766Z DEBUG kafka_starter_rust: RequestHeader request_header=RequestHeader { request_api_key: 1, request_api_version: 16, correlation_id: 933664922, client_id: NullableString("kafka-tester"), tagged_fields: 0 }
[tester::#DH6] Hexdump of sent "Fetch" request:
[tester::#DH6] Idx | Hex | ASCII
[tester::#DH6] -----+-------------------------------------------------+-----------------
[tester::#DH6] 0000 | 00 00 00 30 00 01 00 10 37 a6 98 9a 00 0c 6b 61 | ...0....7.....ka
[tester::#DH6] 0010 | 66 6b 61 2d 74 65 73 74 65 72 00 00 00 01 f4 00 | fka-tester......
[tester::#DH6] 0020 | 00 00 01 03 20 00 00 00 00 00 00 00 00 00 00 00 | .... ...........
[tester::#DH6] 0030 | 01 01 01 00 | ....
[tester::#DH6]
[tester::#DH6] Hexdump of received "Fetch" response:
[tester::#DH6] Idx | Hex | ASCII
[tester::#DH6] -----+-------------------------------------------------+-----------------
[tester::#DH6] 0000 | 37 a6 98 9a 00 00 00 00 00 00 00 00 00 00 00 01 | 7...............
[tester::#DH6] 0010 | 00 | .
[tester::#DH6]
[tester::#DH6] [Decoder] - .ResponseHeader
[tester::#DH6] [Decoder] - .correlation_id (933664922)
[tester::#DH6] [Decoder] - .TAG_BUFFER
[tester::#DH6] [Decoder] - .ResponseBody
[tester::#DH6] [Decoder] - .throttle_time_ms (0)
[tester::#DH6] [Decoder] - .error_code (0)
[tester::#DH6] [Decoder] - .session_id (0)
[tester::#DH6] [Decoder] - .num_responses (0)
[tester::#DH6] [Decoder] - .TAG_BUFFER
[tester::#DH6] ✓ Correlation ID: 933664922
[tester::#DH6] ✓ Throttle Time: 0
[tester::#DH6] ✓ Error Code: 0 (NO_ERROR)
[tester::#DH6] ✓ TopicResponses: []
[tester::#DH6] Test passed.
[tester::#DH6] Terminating program
[tester::#DH6] Program terminated successfully
[tester::#GS0] Running tests for Stage #GS0 (Consuming Messages - Include Fetch in APIVersions)
[tester::#GS0] $ ./your_program.sh /tmp/server.properties
[tester::#GS0] [Serializer] Writing log files to: /tmp/kraft-combined-logs
[tester::#GS0] [Serializer] - Wrote file to: /tmp/server.properties
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/meta.properties
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/.kafka_cleanshutdown
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/foo-0/partition.metadata
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/pax-0/partition.metadata
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/quz-0/partition.metadata
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/quz-1/partition.metadata
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/__cluster_metadata-0/partition.metadata
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/foo-0/00000000000000000000.log
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/pax-0/00000000000000000000.log
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/quz-0/00000000000000000000.log
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/quz-1/00000000000000000000.log
[tester::#GS0] [Serializer] - Wrote file to: /tmp/kraft-combined-logs/__cluster_metadata-0/00000000000000000000.log
[tester::#GS0] [Serializer] Finished writing log files to: /tmp/kraft-combined-logs
[tester::#GS0] Connecting to broker at: localhost:9092
[your_program] Debug mode enabled
[your_program] 2024-10-12T20:04:14.821377Z INFO kafka_starter_rust: Started server listen_addr=127.0.0.1:9092
[tester::#GS0] Connection to broker at localhost:9092 successful
[tester::#GS0] Sending "ApiVersions" (version: 4) request (Correlation id: 1410041126)
[your_program] 2024-10-12T20:04:15.815535Z DEBUG kafka_starter_rust: accepted new connection peer_addr="127.0.0.1:39772"
[tester::#GS0] Hexdump of sent "ApiVersions" request:
[tester::#GS0] Idx | Hex | ASCII
[tester::#GS0] -----+-------------------------------------------------+-----------------
[tester::#GS0] 0000 | 00 00 00 23 00 12 00 04 54 0b 85 26 00 09 6b 61 | ...#....T..&..ka
[tester::#GS0] 0010 | 66 6b 61 2d 63 6c 69 00 0a 6b 61 66 6b 61 2d 63 | fka-cli..kafka-c
[tester::#GS0] 0020 | 6c 69 04 30 2e 31 00 | li.0.1.
[tester::#GS0]
[tester::#GS0] Hexdump of received "ApiVersions" response:
[tester::#GS0] Idx | Hex | ASCII
[tester::#GS0] -----+-------------------------------------------------+-----------------
[tester::#GS0] 0000 | 54 0b 85 26 00 00 00 03 00 12 00 02 00 04 00 00 | T..&............
[tester::#GS0] 0010 | 01 00 0e 00 10 00 00 00 00 00 00 | ...........
[tester::#GS0]
[tester::#GS0] [Decoder] - .ResponseHeader
[tester::#GS0] [Decoder] - .correlation_id (1410041126)
[tester::#GS0] [Decoder] - .ResponseBody
[tester::#GS0] [Decoder] - .error_code (0)
[tester::#GS0] [Decoder] - .num_api_keys (0)
[tester::#GS0] [Decoder] - .throttle_time_ms (50336256)
[tester::#GS0] [Decoder] - .TAG_BUFFER
[your_program] 2024-10-12T20:04:15.815977Z DEBUG kafka_starter_rust: Got frame frame_length=35
[your_program] 2024-10-12T20:04:15.816024Z DEBUG kafka_starter_rust: RequestHeader request_header=RequestHeader { request_api_key: 18, request_api_version: 4, correlation_id: 1410041126, client_id: NullableString("kafka-cli"), tagged_fields: 0 }
[tester::#GS0] Received:
[tester::#GS0] Hex (bytes 15-26) | ASCII
[tester::#GS0] ------------------------------------------------+------------------
[tester::#GS0] 00 01 00 0e 00 10 00 00 00 00 00 00 | ............
[tester::#GS0] ^ ^
[tester::#GS0] Error: unexpected 7 bytes remaining in decoder after decoding ApiVersionsResponse
[tester::#GS0] Context:
[tester::#GS0] - ApiVersions v3
[tester::#GS0] - Response Body
[tester::#GS0]
[tester::#GS0] Test failed