Handshake failed occasionally in #nd2 (download a piece)

[realth000]

The handshake step in stage #nd2 sometimes break when running test online.

If I run test offline with `./your_program.sh download_piece …` it is more likely to get the same error.

I do not know if it’s a network issue, so I post online test log here.

Here are my logs:

-> codecrafters test
Initiating test run...

⚡ This is a turbo test run. https://codecrafters.io/turbo

Running tests. Logs should appear shortly...

[compile]    Compiling codecrafters-bittorrent v0.1.0 (/app)
[compile]     Finished `release` profile [optimized] target(s) in 37.71s
[compile] Moved ./.codecrafters/run.sh → ./your_program.sh
[compile] Compilation successful.

[tester::#ND2] Running tests for Stage #ND2 (Download a piece)
[tester::#ND2] Running ./your_program.sh download_piece -o /tmp/torrents3401819537/piece-2 /tmp/torrents3401819537/congratulations.gif.torrent 2
[your_program] >>> handshake: ip=165.232.41.73, port=51549
[your_program] >>> handshake request: [19, 66, 105, 116, 84, 111, 114, 114, 101, 110, 116, 32, 112, 114, 111, 116, 111, 99, 111, 108, 0, 0, 0, 0, 0, 0, 0, 0, 28, 173, 74, 72, 103, 152, 217, 82, 97, 76, 57, 78, 177, 94, 117, 190, 197, 135, 253, 8, 108, 49, 53, 52, 114, 75, 113, 79, 72, 107, 102, 77, 76, 69, 71, 65, 101, 99, 101, 121]
[your_program] Error: invalid resp message format
[your_program]
[your_program] Caused by:
[your_program]     warning: invalid handshake message length: 74, data=[19, 66, 105, 116, 84, 111, 114, 114, 101, 110, 116, 32, 112, 114, 111, 116, 111, 99, 111, 108, 0, 0, 0, 0, 0, 0, 0, 4, 28, 173, 74, 72, 103, 152, 217, 82, 97, 76, 57, 78, 177, 94, 117, 190, 197, 135, 253, 8, 45, 82, 78, 48, 46, 48, 46, 48, 45, 246, 253, 67, 135, 224, 214, 243, 189, 22, 99, 239, 0, 0, 0, 2, 5, 240]
[tester::#ND2] Application didn't terminate successfully without errors. Expected 0 as exit code, got: 1
[tester::#ND2] Test failed (try setting 'debug: true' in your codecrafters.yml to see more details)

View our article on debugging test failures: https://codecrafters.io/debug

The handshake response is not 68 bytes long, and the reserved 8 bytes are `0 0 0 0 0 0 0 4`, I found the same value in the first picture here.

And here’s a snippet of my code:

#[derive(Debug)]
pub struct HandshakeMessage {
    /// Sha1 info hash.
    pub info_hash: [u8; 20],

    /// Peer id in byte array.
    pub peer_id: [u8; 20],
}

impl HandshakeMessage {
    pub fn new(info_hash: [u8; 20], peer_id: [u8; 20]) -> Self {
        Self { info_hash, peer_id }
    }

    pub fn from_bytes(buffer: &[u8]) -> Result<Self> {
        if buffer.len() != 1 + 19 + 8 + 20 + 20 {
            bail!(
                "warning: invalid handshake message length: {}, data={:?}",
                buffer.len(),
                buffer,
            )
        }
        const HEADER_LEN: usize = 1 + 19 + 8;
        // TODO: Check header.
        let info_hash = buffer[HEADER_LEN..HEADER_LEN + 20]
            .iter()
            .map(|x| x.to_owned().to_owned())
            .collect::<Vec<_>>()
            .try_into()
            .unwrap();
        let peer_id = buffer[HEADER_LEN + 20..HEADER_LEN + 20 + 20]
            .iter()
            .map(|x| x.to_owned().to_owned())
            .collect::<Vec<_>>()
            .try_into()
            .unwrap();
        Ok(Self { info_hash, peer_id })
    }

    fn to_bytes(&self) -> Vec<u8> {
        let mut buffer = Vec::with_capacity(128);
        buffer.push(19);
        buffer.extend_from_slice(b"BitTorrent protocol");
        buffer.extend_from_slice(&[0u8; 8]);
        buffer.extend_from_slice(self.info_hash.as_slice());
        buffer.extend_from_slice(self.peer_id.as_slice());
        buffer
    }
}

pub async fn handshake(
    ip: &str,
    port: u16,
    message: HandshakeMessage,
) -> BtResult<HandshakeMessage> {
    let mut socket = TcpStream::connect(format!("{ip}:{port}").as_str())
        .await
        .context("failed to dial")?;
    let (mut rd, mut wr) = socket.split();
    if let Err(e) = wr.write_all(&message.to_bytes()).await {
        bail!("failed to send handshake message: {e}")
    }

    let mut buf = vec![0; 256];

    loop {
        let n = rd.read(&mut buf).await?;
        if n == 0 {
            break;
        }
        let resp =
            HandshakeMessage::from_bytes(&buf[0..n]).context("invalid resp message format")?;
        return Ok(resp);
    }

    bail!("empty responce");
}

Full code here: codecrafters-bittorrent-rust/src/http.rs at b4198e8aeeacc09369d812d6b934c5ef6a84da39 · realth000/codecrafters-bittorrent-rust · GitHub

BTW, after long time debugging I’m not sure if the block data is correct, result told me the SHA-1 hash is incorrect, but the handshake issue is stopping me test my code offline.

[andy1li]

Hey @realth000, there might be a few issues at play. The first thing I noticed is the hash did not match:

I added a couple of logs here:

What’s odd is that blk_buf ended up larger than expected:

Could you try replacing the read loop with read_exact? That should take care of the hash issue once blk_buf is read properly.

Let me know how it goes, and we can move forward from there.

[realth000]

Thanks! Everything works well when using read_exact, both handshake and SHA-1 hash, now I can test my program locally.