Stage #WA6 (Test failed: read: connection reset by peer)

Challenges
1

I’m stuck on Stage #WA6.

I’ve tried running this locally and using print debug statements; all looks fine. The code from the previous task is essentially the same, and I’m outputting the expected correlation ID.

Here are my logs:

[compile]    Compiling codecrafters-kafka v0.1.0 (/app)
[compile]     Finished `release` profile [optimized] target(s) in 16.76s
[compile] Moved ./.codecrafters/run.sh → ./your_program.sh
[compile] Compilation successful.
Debug = true
[tester::#WA6] Running tests for Stage #WA6 (Parse Correlation ID)
[tester::#WA6] $ ./your_program.sh /tmp/server.properties
[tester::#WA6] Connecting to broker at: localhost:9092
[your_program] Logs from your program will appear here!
[your_program] accepted new connection
[your_program] Buffer length: 12
[tester::#WA6] Connection to broker at localhost:9092 successful
[tester::#WA6] Sending "ApiVersions" (version: 4) request (Correlation id: 1699631701)
[tester::#WA6] Hexdump of sent "ApiVersions" request: 
[tester::#WA6] Idx  | Hex                                             | ASCII
[tester::#WA6] -----+-------------------------------------------------+-----------------
[tester::#WA6] 0000 | 00 00 00 23 00 12 00 04 65 4e 52 55 00 09 6b 61 | ...#....eNRU..ka
[tester::#WA6] 0010 | 66 6b 61 2d 63 6c 69 00 0a 6b 61 66 6b 61 2d 63 | fka-cli..kafka-c
[tester::#WA6] 0020 | 6c 69 04 30 2e 31 00                            | li.0.1.
[tester::#WA6] 
[your_program] Read 12 byte(s)
[tester::#WA6] error reading from connection: read tcp 127.0.0.1:39756->127.0.0.1:9092: read: connection reset by peer
[tester::#WA6] Test failed
[tester::#WA6] Terminating program
[your_program] Total bytes read: 12
[your_program] Correlation ID: 1699631701
[your_program] Sent 4 byte(s) for message size
[your_program] Sent 4 byte(s) for correlation ID
[tester::#WA6] Program terminated successfully

The main error seems to be this:

error reading from connection: read tcp 127.0.0.1:39756->127.0.0.1:9092: read: connection reset by peer

And here’s a snippet of my code:

fn read_bytes_from_stream(_stream: &mut TcpStream, buf: &mut [u8]) -> usize {
    let mut total_bytes_read = 0;
    println!("Buffer length: {}", buf.len());
    while total_bytes_read < buf.len() {
        match _stream.read(&mut buf[total_bytes_read..]) {
            Ok(0) => {
                println!("Connection closed by peer");
                break
            },
            Ok(n) => {
                println!("Read {} byte(s)", n);
                total_bytes_read += n;
            },
            Err(e) => {
                eprintln!("Failed to read: {}", e);
                break
            },
        }
    }
    println!("Total bytes read: {}", total_bytes_read);
    total_bytes_read
}

fn parse_correlation_id(bytes: &[u8], offset: usize, size: usize) -> i32 {
    let correlation_id = i32::from_be_bytes(bytes[offset..size].try_into().unwrap());
    correlation_id
}

The rest can be found here.

Thanks for your help.

4

Connection reset I believe generally occurs when the connection is closed before the full request has been read. I see you only read up to correlation id/ hardcode the message size, but BinspecVisualizer suggests the message can be longer and maybe of variable length.

1 Like
5

Thanks for your help.

I’ve tried reading all the data at once with read_to_end and iteratively using read until no more is readable, but this initially caused the same error.

Now, a new error is displayed (see log below) that seems to suggest the client closes the connection before the server/broker code can write the response message. This may be a half-closed connection, but it doesn’t seem to be from the server/broker end. It seems to be failing during or just after the read.

[compile]    Compiling codecrafters-kafka v0.1.0 (/app)
[compile]     Finished `release` profile [optimized] target(s) in 16.28s
[compile] Moved ./.codecrafters/run.sh → ./your_program.sh
[compile] Compilation successful.
Debug = true
[tester::#WA6] Running tests for Stage #WA6 (Parse Correlation ID)
[tester::#WA6] $ ./your_program.sh /tmp/server.properties
[tester::#WA6] Connecting to broker at: localhost:9092
[your_program] Logs from your program will appear here!
[tester::#WA6] Connection to broker at localhost:9092 successful
[tester::#WA6] Sending "ApiVersions" (version: 4) request (Correlation id: 495553751)
[tester::#WA6] Hexdump of sent "ApiVersions" request: 
[tester::#WA6] Idx  | Hex                                             | ASCII
[tester::#WA6] -----+-------------------------------------------------+-----------------
[tester::#WA6] 0000 | 00 00 00 23 00 12 00 04 1d 89 8c d7 00 09 6b 61 | ...#..........ka
[tester::#WA6] 0010 | 66 6b 61 2d 63 6c 69 00 0a 6b 61 66 6b 61 2d 63 | fka-cli..kafka-c
[tester::#WA6] 0020 | 6c 69 04 30 2e 31 00                            | li.0.1.
[tester::#WA6] 
[your_program] 
[your_program] Accepted new connection
[your_program] Read 39 byte(s)
[tester::#WA6] Hexdump of received "ApiVersions" response: 
[tester::#WA6] Idx  | Hex                                             | ASCII
[tester::#WA6] -----+-------------------------------------------------+-----------------
[tester::#WA6] | 
[tester::#WA6] 
[tester::#WA6] [Decoder] - .Response
[tester::#WA6] [Decoder] Received:
[tester::#WA6] [Decoder] Hex (bytes 0--1)                                | ASCII
[tester::#WA6] [Decoder] ------------------------------------------------+------------------
[tester::#WA6] [Decoder] | 
[tester::#WA6] [Decoder]  ^                                                ^
[tester::#WA6] [Decoder] Error: Expected int32 length to be 4 bytes, got 0 bytes
[tester::#WA6] [Decoder] Context:
[tester::#WA6] [Decoder] - response
[tester::#WA6] [Decoder]   - message length
[tester::#WA6] [Decoder]     - INT32
[tester::#WA6] [Decoder] 
[tester::#WA6] [Decoder] Test failed
[tester::#WA6] [Decoder] Terminating program
[your_program] Connection closed by peer
[your_program] Total bytes read: 39
[your_program] Correlation ID: 495553751
[tester::#WA6] [Decoder] Program terminated successfully
6

I was reading from the stream then writing, but by then the connection might have been half-closed (probably from the client/peer side), so it wasn’t able to write the bytes in time.

The solution was to write the bytes before the last read attempt (i.e. zero bytes read/ and [read] connection closed by peer).

7

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.