Stage #WA6 (Test failed: read: connection reset by peer)

I’m stuck on Stage #WA6.

I’ve tried running this locally and using print debug statements; all looks fine. The code from the previous task is essentially the same, and I’m outputting the expected correlation ID.

Here are my logs:

[compile]    Compiling codecrafters-kafka v0.1.0 (/app)
[compile]     Finished `release` profile [optimized] target(s) in 16.76s
[compile] Moved ./.codecrafters/run.sh → ./your_program.sh
[compile] Compilation successful.
Debug = true
[tester::#WA6] Running tests for Stage #WA6 (Parse Correlation ID)
[tester::#WA6] $ ./your_program.sh /tmp/server.properties
[tester::#WA6] Connecting to broker at: localhost:9092
[your_program] Logs from your program will appear here!
[your_program] accepted new connection
[your_program] Buffer length: 12
[tester::#WA6] Connection to broker at localhost:9092 successful
[tester::#WA6] Sending "ApiVersions" (version: 4) request (Correlation id: 1699631701)
[tester::#WA6] Hexdump of sent "ApiVersions" request: 
[tester::#WA6] Idx  | Hex                                             | ASCII
[tester::#WA6] -----+-------------------------------------------------+-----------------
[tester::#WA6] 0000 | 00 00 00 23 00 12 00 04 65 4e 52 55 00 09 6b 61 | ...#....eNRU..ka
[tester::#WA6] 0010 | 66 6b 61 2d 63 6c 69 00 0a 6b 61 66 6b 61 2d 63 | fka-cli..kafka-c
[tester::#WA6] 0020 | 6c 69 04 30 2e 31 00                            | li.0.1.
[tester::#WA6] 
[your_program] Read 12 byte(s)
[tester::#WA6] error reading from connection: read tcp 127.0.0.1:39756->127.0.0.1:9092: read: connection reset by peer
[tester::#WA6] Test failed
[tester::#WA6] Terminating program
[your_program] Total bytes read: 12
[your_program] Correlation ID: 1699631701
[your_program] Sent 4 byte(s) for message size
[your_program] Sent 4 byte(s) for correlation ID
[tester::#WA6] Program terminated successfully

The main error seems to be this:

error reading from connection: read tcp 127.0.0.1:39756->127.0.0.1:9092: read: connection reset by peer

And here’s a snippet of my code:

fn read_bytes_from_stream(_stream: &mut TcpStream, buf: &mut [u8]) -> usize {
    let mut total_bytes_read = 0;
    println!("Buffer length: {}", buf.len());
    while total_bytes_read < buf.len() {
        match _stream.read(&mut buf[total_bytes_read..]) {
            Ok(0) => {
                println!("Connection closed by peer");
                break
            },
            Ok(n) => {
                println!("Read {} byte(s)", n);
                total_bytes_read += n;
            },
            Err(e) => {
                eprintln!("Failed to read: {}", e);
                break
            },
        }
    }
    println!("Total bytes read: {}", total_bytes_read);
    total_bytes_read
}

fn parse_correlation_id(bytes: &[u8], offset: usize, size: usize) -> i32 {
    let correlation_id = i32::from_be_bytes(bytes[offset..size].try_into().unwrap());
    correlation_id
}

The rest can be found here.

Thanks for your help.

Connection reset I believe generally occurs when the connection is closed before the full request has been read. I see you only read up to correlation id/ hardcode the message size, but BinspecVisualizer suggests the message can be longer and maybe of variable length.

Thanks for your help.

I’ve tried reading all the data at once with read_to_end and iteratively using read until no more is readable, but this initially caused the same error.

Now, a new error is displayed (see log below) that seems to suggest the client closes the connection before the server/broker code can write the response message. This may be a half-closed connection, but it doesn’t seem to be from the server/broker end. It seems to be failing during or just after the read.

[compile]    Compiling codecrafters-kafka v0.1.0 (/app)
[compile]     Finished `release` profile [optimized] target(s) in 16.28s
[compile] Moved ./.codecrafters/run.sh → ./your_program.sh
[compile] Compilation successful.
Debug = true
[tester::#WA6] Running tests for Stage #WA6 (Parse Correlation ID)
[tester::#WA6] $ ./your_program.sh /tmp/server.properties
[tester::#WA6] Connecting to broker at: localhost:9092
[your_program] Logs from your program will appear here!
[tester::#WA6] Connection to broker at localhost:9092 successful
[tester::#WA6] Sending "ApiVersions" (version: 4) request (Correlation id: 495553751)
[tester::#WA6] Hexdump of sent "ApiVersions" request: 
[tester::#WA6] Idx  | Hex                                             | ASCII
[tester::#WA6] -----+-------------------------------------------------+-----------------
[tester::#WA6] 0000 | 00 00 00 23 00 12 00 04 1d 89 8c d7 00 09 6b 61 | ...#..........ka
[tester::#WA6] 0010 | 66 6b 61 2d 63 6c 69 00 0a 6b 61 66 6b 61 2d 63 | fka-cli..kafka-c
[tester::#WA6] 0020 | 6c 69 04 30 2e 31 00                            | li.0.1.
[tester::#WA6] 
[your_program] 
[your_program] Accepted new connection
[your_program] Read 39 byte(s)
[tester::#WA6] Hexdump of received "ApiVersions" response: 
[tester::#WA6] Idx  | Hex                                             | ASCII
[tester::#WA6] -----+-------------------------------------------------+-----------------
[tester::#WA6] | 
[tester::#WA6] 
[tester::#WA6] [Decoder] - .Response
[tester::#WA6] [Decoder] Received:
[tester::#WA6] [Decoder] Hex (bytes 0--1)                                | ASCII
[tester::#WA6] [Decoder] ------------------------------------------------+------------------
[tester::#WA6] [Decoder] | 
[tester::#WA6] [Decoder]  ^                                                ^
[tester::#WA6] [Decoder] Error: Expected int32 length to be 4 bytes, got 0 bytes
[tester::#WA6] [Decoder] Context:
[tester::#WA6] [Decoder] - response
[tester::#WA6] [Decoder]   - message length
[tester::#WA6] [Decoder]     - INT32
[tester::#WA6] [Decoder] 
[tester::#WA6] [Decoder] Test failed
[tester::#WA6] [Decoder] Terminating program
[your_program] Connection closed by peer
[your_program] Total bytes read: 39
[your_program] Correlation ID: 495553751
[tester::#WA6] [Decoder] Program terminated successfully

I was reading from the stream then writing, but by then the connection might have been half-closed (probably from the client/peer side), so it wasn’t able to write the bytes in time.

The solution was to write the bytes before the last read attempt (i.e. zero bytes read/ and [read] connection closed by peer).