Unable to receive bitfield message

[malay44]

I’m stuck on Stage #ND2.

code runs smoothly on my machine, but fails during test.
This was not the case few days back, when I was debugging the issue with hash matching of downloaded piece

Here are my logs:

Test logs

\codecrafters-bittorrent-go master ×1​:railway_track: ×1�� ×2via v1.22.0 5GiB/8GiB took 1m37s
❯ codecrafters test
Initiating test run…

Running tests. Logs should appear shortly…

[compile] Compilation successful.

Debug = true

[tester::#ND2] Running tests for Stage #ND2 (Download a piece)
[tester::#ND2] Running ./your_bittorrent.sh download_piece -o /tmp/torrents329709923/piece-9 /tmp/torrents329709923/itsworking.gif.torrent 9
[your_program] Starting retrieval for piece 9 from torrent file /tmp/torrents329709923/itsworking.gif.torrent
[your_program] Parsed torrent file successfully
[your_program] Tracker URL: http://bittorrent-test-tracker.codecrafters.io/announce
[your_program] Length: 2549700
[your_program] Info Hash: 70edcac2611a8829ebf467a6849f5d8408d9d8f4
[your_program] Piece Length: 262144
[your_program] Piece Hashes:
[your_program] 01cc17bbe60fa5a52f64bd5f5b64d99286d50aa5
[your_program] 838f703cf7f6f08d1c497ed390df78f90d5f7566
[your_program] 45bf10974b5816491e30628b78a382ca36c4e05f
[your_program] 84be4bd855b34bcedc0c6e98f66d3e7c63353d1e
[your_program] 86427ac94d6e4f21a6d0d6c8b7ffa4c393c3b131
[your_program] 7c70cd5f44d1ac5505cb855d526ceb0f5f1cd5e3
[your_program] 3796ab05af1fa874173a0a6c1298625ad47b4fe6
[your_program] 272a8ff8fc865b053d974a78681414b38077d7b1
[your_program] b07128d3a6018062bfe779db96d3a93c05fb81d4
[your_program] 7affc94f0985b985eb888a36ec92652821a21be4
[your_program] Retrieved peer list: [161.35.46.221:51520 159.65.84.183:51423 167.172.57.188:51544]
[your_program] Handshake with peer successful
[your_program] Waiting for message with ID 5
[your_program] Error reading length prefix: EOF
[your_program] Error waiting for bitfield message: EOF
[your_program] connection closed
[your_program] EOF
[tester::#ND2] Connection reset by peer or EOF type of errors can be transient issues (try again), but they can also happen when peers receive unexpected messages (sending a REQUEST message with an incorrect offset/length, block size larger than 16 kb etc.)
[tester::#ND2] Last piece of the file can be less than piece length specified in torrent metadata. For instance, the length of this piece is 190404
[tester::#ND2] File does not exist: /tmp/torrents329709923/piece-9
[tester::#ND2] Test failed

Local logs

codecrafters-bittorrent-go master ×1​:railway_track: ×1�� ×2via v1.22.0 6GiB/8GiB took 10s
❯ ./your_bittorrent.sh download_piece -o ./tmp/piece-9 ./itsworking.gif.torrent 9
Starting retrieval for piece 9 from torrent file ./itsworking.gif.torrent
Parsed torrent file successfully
Tracker URL: http://bittorrent-test-tracker.codecrafters.io/announce
Length: 2549700
Info Hash: 70edcac2611a8829ebf467a6849f5d8408d9d8f4
Piece Length: 262144
Piece Hashes:
01cc17bbe60fa5a52f64bd5f5b64d99286d50aa5
838f703cf7f6f08d1c497ed390df78f90d5f7566
45bf10974b5816491e30628b78a382ca36c4e05f
84be4bd855b34bcedc0c6e98f66d3e7c63353d1e
86427ac94d6e4f21a6d0d6c8b7ffa4c393c3b131
7c70cd5f44d1ac5505cb855d526ceb0f5f1cd5e3
3796ab05af1fa874173a0a6c1298625ad47b4fe6
272a8ff8fc865b053d974a78681414b38077d7b1
b07128d3a6018062bfe779db96d3a93c05fb81d4
7affc94f0985b985eb888a36ec92652821a21be4
Retrieved peer list: [161.35.46.221:51520 159.65.84.183:51423 167.172.57.188:51544]
Handshake with peer successful
Waiting for message with ID 5
Received payload with ID 5, length 2
Received bitfield message
Sending interested message to peer
Waiting for message with ID 1
Received payload with ID 1, length 0
Received unchoke message
payload: [0 0 0 9 0 0 0 0 0 0 64 0]
Sending request for block at offset 0 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 0 64 0 0 0 64 0]
Sending request for block at offset 16384 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 0 128 0 0 0 64 0]
Sending request for block at offset 32768 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 0 192 0 0 0 64 0]
Sending request for block at offset 49152 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 1 0 0 0 0 64 0]
Sending request for block at offset 65536 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 1 64 0 0 0 64 0]
Sending request for block at offset 81920 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 1 128 0 0 0 64 0]
Sending request for block at offset 98304 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 1 192 0 0 0 64 0]
Sending request for block at offset 114688 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 2 0 0 0 0 64 0]
Sending request for block at offset 131072 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 2 64 0 0 0 64 0]
Sending request for block at offset 147456 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 2 128 0 0 0 64 0]
Sending request for block at offset 163840 (length: 16384)
Created message with ID 6 and payload length 12
payload: [0 0 0 9 0 2 192 0 0 0 39 196]
Sending request for block at offset 180224 (length: 10180)
Created message with ID 6 and payload length 12
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 0–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 16384–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 32768–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 49152–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 65536–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 81920–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 98304–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 114688–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 131072–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 147456–
Waiting for message with ID 7
Received payload with ID 7, length 16392
–Received block data for offset 163840–
Waiting for message with ID 7
Received payload with ID 7, length 10188
–Received block data for offset 180224–
Piece hash: 7affc94f0985b985eb888a36ec92652821a21be4
Torrent hash: 7affc94f0985b985eb888a36ec92652821a21be4
Piece hash verified successfully, writing data to file
Data written to file successfully at ./tmp/piece-9
Piece retrieval successful
connection closed

[malay44]

And here’s my github repo link:- GitHub - malay44/ocTorrent

[malay44]

Local Run Logs

[malay44]

Test Logs

[andy1li]

@malay44 I added a log as shown below and ran your code:

It seems that the handshake did not succeed:

We’re currently investigating the handshake issue. While it may take some time, I’ll keep you updated once we have a resolution.

[sarp]

Hi malay44,
There are currently two issues that might be causing this:

1. Peers will not accept multiple connections with the same peer_id

Can you try again using a random peer_id instead of 00112233445566778899 to reduce the likelihood of colliding with another Codecrafters user.

2. Peers will not accept multiple connections with the same IP address

Our test runners can share same outgoing IP address, we’re working on a fix to address this issue. If it’s working locally but not on codecrafters, this might be related. As Andy mentioned, we’ll update once we have a resolution.

[malay44]

tried using other id (abzxbwkqvglwhrpvfnmx) but it still fails in test

[andy1li]

@malay44 Could you give it another try and please let me know how it goes?

@rohitpaulk

Hey everyone! We’ve pushed an update that should remove the IP-based rate-limiting. Thanks to @sarp for making this change!

I’ve also changed our E2E tests to specifically test for this - earlier they weren’t trying to download the same file in parallel, so we weren’t hitting this failure case.

[malay44]

it’s working, thanks a lot guys