Hello,
When creating the response to the client, should we consider a specific order like correlation id comes before api version ? As we are parsing the request using api_version before correlation_id, I expected the same to happen in the response side.
Please can someone clarify this point.
There is no way you can parse a binary stream and arrange the order. The order is fixed and must be known so that when you receive the stream you know what each byte represents.
Computer programs (including Kafka) that communicate over networks use protocols, which are essentially predefined rules. Unfortunately, these rules may not always align with our expectations.
Take greetings in Arabic for example:
Someone might expect the words in the “request” and “response” to be in the same order, but that’s not necessarily how people (or Kafka) communicate.
Thanks for the clarification and the example @andy1li ! However wouldn’t that raise inconsistency in the way Kafka handles communications ?
I mean The protocol has a strict format and predefined rules that must be followed, otherwise, the communication would fail right ?
Thanks @lfmunoz for the response, in our case from where can I know the order that the response needs to follow, did I miss this in the task ?
As I read the documentation, the correlation id seems to come after the api_version.
Everything is specified here:
But as you noticed it isn’t easy to understand.
It shows:
RequestOrResponse => Size (RequestMessage | ResponseMessage)
Size => int32
Request Header v0 => request_api_key request_api_version correlation_id
request_api_key => INT16
request_api_version => INT16
correlation_id => INT32
What this means is :
You receive a “RequestMessage” o
the first 4 bytes are the length, you must grab that amount of bytes
You receive a “Request Header”
the next 2 bytes identify the type of packet “request_api_key”
There is a huge list on that document of what these represement:
APIVersions(18),
DescribeTopicPartitions(75);
the next two bytes are “request_api_version” of the “request_api_key”
then next 4 are correlation_id
after than you will get the actual packet so request_api_key=18 that is APIVersions so looking at that part of the document
ApiVersions Request (Version: 0) =>
so for request_api_version = 0 and request_api_key = 18. that is it.
You are then expected to send a response. so you look at response section
It shows:
ApiVersions Response (Version: 1) => error_code [api_keys] throttle_time_ms
error_code => INT16
api_keys => api_key min_version max_version
api_key => INT16
min_version => INT16
max_version => INT16
throttle_time_ms => INT32
and you start sending that back.
It starts becoming tedious.
Short answer, no.
Requests follow the predefined rules for requests, and responses follow the predefined rules for responses. The rules don’t necessarily have to be the same for different types of communication.
Yes, that’s correct!
Feel free to discuss any specific inconsistencies you’ve noticed. 
Closing this thread due to inactivity. If you still need assistance, feel free to reopen or start a new discussion!
This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.