JV8 timing out when downloading a file

Similar to: Execution time out on test #JV8

My program times out in the CodeCrafter runner, even though it says downloading from multiple peers at once is optional.

Currently my code is synchronous, as it waits for a piece to be downloaded before downloading the next one. So, is it in fact mandatory to download blocks/pieces concurrently for this task?

My execution logs:

remote: [tester::#JV8] Running tests for Stage #JV8 (Download the whole file)
remote: [tester::#JV8] Running ./your_bittorrent.sh download -o /tmp/torrents1341413538/codercat.gif /tmp/torrents1341413538/codercat.gif.torrent
remote: [your_program] Total number of pieces: 12
remote: [your_program] Peer ID: 2d524e302e302e302d89d20d086fea9fcedc2c9d
remote: [your_program] Waiting for bitfield message...
remote: [your_program] Received message with ID 5.
remote: [your_program] 🫸🏻 Waiting for unchoke message...
remote: [your_program] Received message with ID 1.
remote: [your_program] 📥 Received unchoke message.
remote: [your_program] Requesting block 1 of 16 with length 16384
remote: [your_program] Requesting block 2 of 16 with length 16384
remote: [your_program] Requesting block 3 of 16 with length 16384
remote: [your_program] Requesting block 4 of 16 with length 16384
remote: [your_program] Requesting block 5 of 16 with length 16384
remote: [your_program] Requesting block 6 of 16 with length 16384
remote: [your_program] Requesting block 7 of 16 with length 16384
remote: [your_program] Requesting block 8 of 16 with length 16384
remote: [your_program] Requesting block 9 of 16 with length 16384
remote: [your_program] Requesting block 10 of 16 with length 16384
remote: [your_program] Requesting block 11 of 16 with length 16384
remote: [your_program] Requesting block 12 of 16 with length 16384
remote: [your_program] Requesting block 13 of 16 with length 16384
remote: [your_program] Requesting block 14 of 16 with length 16384
remote: [your_program] Requesting block 15 of 16 with length 16384
remote: [your_program] Requesting block 16 of 16 with length 16384
remote: [your_program] Peer ID: 2d524e302e302e302d89d20d086fea9fcedc2c9d
remote: [your_program] Waiting for bitfield message...
remote: [your_program] Received message with ID 5.
remote: [your_program] 🫸🏻 Waiting for unchoke message...
remote: [your_program] Received message with ID 1.
remote: [your_program] 📥 Received unchoke message.
remote: [your_program] Requesting block 1 of 16 with length 16384
remote: [your_program] Requesting block 2 of 16 with length 16384
remote: [your_program] Requesting block 3 of 16 with length 16384
remote: [your_program] Requesting block 4 of 16 with length 16384
remote: [your_program] Requesting block 5 of 16 with length 16384
remote: [your_program] Requesting block 6 of 16 with length 16384
remote: [your_program] Requesting block 7 of 16 with length 16384
remote: [your_program] Requesting block 8 of 16 with length 16384
remote: [your_program] Requesting block 9 of 16 with length 16384
remote: [your_program] Requesting block 10 of 16 with length 16384
remote: [your_program] Requesting block 11 of 16 with length 16384
remote: [your_program] Requesting block 12 of 16 with length 16384
remote: [your_program] Requesting block 13 of 16 with length 16384
remote: [your_program] Requesting block 14 of 16 with length 16384
remote: [your_program] Requesting block 15 of 16 with length 16384
remote: [your_program] Requesting block 16 of 16 with length 16384
remote: [your_program] Peer ID: 2d524e302e302e302d89d20d086fea9fcedc2c9d
remote: [your_program] Waiting for bitfield message...
remote: [your_program] Received message with ID 5.
remote: [your_program] 🫸🏻 Waiting for unchoke message...
remote: [your_program] Received message with ID 1.
remote: [your_program] 📥 Received unchoke message.
remote: [your_program] Requesting block 1 of 16 with length 16384
remote: [your_program] Requesting block 2 of 16 with length 16384
remote: [your_program] Requesting block 3 of 16 with length 16384
remote: [your_program] Requesting block 4 of 16 with length 16384
remote: [your_program] Requesting block 5 of 16 with length 16384
remote: [your_program] Requesting block 6 of 16 with length 16384
remote: [your_program] Requesting block 7 of 16 with length 16384
remote: [your_program] Requesting block 8 of 16 with length 16384
remote: [your_program] Requesting block 9 of 16 with length 16384
remote: [your_program] Requesting block 10 of 16 with length 16384
remote: [your_program] Requesting block 11 of 16 with length 16384
remote: [your_program] Requesting block 12 of 16 with length 16384
remote: [your_program] Requesting block 13 of 16 with length 16384
remote: [your_program] Requesting block 14 of 16 with length 16384
remote: [your_program] Requesting block 15 of 16 with length 16384
remote: [your_program] Requesting block 16 of 16 with length 16384
remote: [tester::#JV8] execution timed out
remote: [tester::#JV8] Test failed

My code: GitHub - merc1er/bittorrent-python

From my experience, kinda. The test will pick a GIF at random to download from a set of 3 if I recall. Your logs state that you tried it with codercat.gif.torrent, which I think is the largest of the 3.

My synchronous implementation only worked with congratulations.gif.torrent (consistently). Even downloading from multiple peers at once didn’t cut it for me.

Once I changed to an async implementation with request pipelining I started getting consistent test results for all gifs. So while it was possible to get the tests passing at first, I only felt the stage was actually complete once I went full async (also I didn’t want to progress with flaky tests).

2 Likes

Thank you! I will migrate to an async implementation then.

Do you know if it is possible to download multiple blocks from the same peer at once? Or what is the recommendation? I saw 5 peers/connections at once somewhere in the challenge. So maybe that would be the sweet spot?

What I ended up doing is assigning each piece to a different peer concurrently.

2 Likes

In my case I ended up doing both things:

  1. Assigned each piece to a peer concurrently
  2. Chunked pieces and downloaded each chunk concurrently. I didn’t use a magic number of 5 or anything, but I did have to use a semaphore to avoid race conditions when receiving/reading responses.

Just 1. didn’t cut it for me, I was still getting intermittent test failures. Once I nailed 2. tests were green 100% of the time.

2 Likes

I am getting exactly the same thing: my tests are sometimes working, sometimes not, and I only implemented 1.

I will implement 2 as well, since it is making my subsequent tests for the magnet link fail.

Thank you so much for your help!

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.