Command processing #YG4 - Sporadic test results

I completed the command processing #YG4, but during testing, I get sporadic test pass/failures.
Sometimes it’s a "connection refused "error, other times a “no content received” error on a GET key request. All tests pass at other times. I suspect this could be a limitation of the test environment or TCP in general, but I want confirmation if it’s not due to my code. Did everyone have the same experience?

Note: I implemented a custom event loop using mio rather than threads or Tokio in Rust, which may affect timing behavior.

Here are my “connection refused” test failure logs:

[tester::#YG4] Running tests for Stage #YG4 (Replication - Command processing)
[tester::#YG4] Master is running on port 6379
[tester::#YG4] $ ./your_program.sh --port 6380 --replicaof "localhost 6379"
[your_program] Logs from your program will appear here!
[tester::#YG4] [handshake] [master] Waiting for replica to initiate handshake with "PING" command
[tester::#YG4] [handshake] [master] ✔︎ Received ["PING"]
[tester::#YG4] [handshake] [master] Sent "PONG"
[tester::#YG4] [handshake] [master] Waiting for replica to send "REPLCONF listening-port 6380" command
[tester::#YG4] [handshake] [master] ✔︎ Received [
[tester::#YG4] [handshake] [master]   "REPLCONF",
[tester::#YG4] [handshake] [master]   "listening-port",
[tester::#YG4] [handshake] [master]   "6380"
[tester::#YG4] [handshake] [master] ]
[tester::#YG4] [handshake] [master] Sent "OK"
[tester::#YG4] [handshake] [master] Waiting for replica to send "REPLCONF capa" command
[tester::#YG4] [handshake] [master] ✔︎ Received ["REPLCONF", "capa", "psync2"]
[tester::#YG4] [handshake] [master] Sent "OK"
[tester::#YG4] [handshake] [master] Waiting for replica to send "PSYNC" command
[tester::#YG4] [handshake] [master] ✔︎ Received ["PSYNC", "?", "-1"]
[tester::#YG4] [handshake] [master] Sent "FULLRESYNC 75cd7bc10c49047e0d163660f3b90625b1af31dc 0"
[tester::#YG4] [handshake] [master] Sent RDB file.
[tester::#YG4] dial tcp 127.0.0.1:6380: connect: connection refused
[tester::#YG4] Test failed

Here are my “no content received” test failure logs:

[tester::#YG4] Running tests for Stage #YG4 (Replication - Command processing)
[tester::#YG4] Master is running on port 6379
[tester::#YG4] $ ./your_program.sh --port 6380 --replicaof "localhost 6379"
[your_program] Logs from your program will appear here!
[tester::#YG4] [handshake] [master] Waiting for replica to initiate handshake with "PING" command
[tester::#YG4] [handshake] [master] ✔︎ Received ["PING"]
[tester::#YG4] [handshake] [master] Sent "PONG"
[tester::#YG4] [handshake] [master] Waiting for replica to send "REPLCONF listening-port 6380" command
[tester::#YG4] [handshake] [master] ✔︎ Received [
[tester::#YG4] [handshake] [master]   "REPLCONF",
[tester::#YG4] [handshake] [master]   "listening-port",
[tester::#YG4] [handshake] [master]   "6380"
[tester::#YG4] [handshake] [master] ]
[tester::#YG4] [handshake] [master] Sent "OK"
[tester::#YG4] [handshake] [master] Waiting for replica to send "REPLCONF capa" command
[tester::#YG4] [handshake] [master] ✔︎ Received ["REPLCONF", "capa", "psync2"]
[tester::#YG4] [handshake] [master] Sent "OK"
[tester::#YG4] [handshake] [master] Waiting for replica to send "PSYNC" command
[tester::#YG4] [handshake] [master] ✔︎ Received ["PSYNC", "?", "-1"]
[tester::#YG4] [handshake] [master] Sent "FULLRESYNC 75cd7bc10c49047e0d163660f3b90625b1af31dc 0"
[tester::#YG4] [handshake] [master] Sent RDB file.
[tester::#YG4] [propagation] [master] > SET foo 123
[tester::#YG4] [propagation] [master] > SET bar 456
[tester::#YG4] [propagation] [master] > SET baz 789
[your_program] new connection from 127.0.0.1:36900
[tester::#YG4] [test] Getting key foo
[tester::#YG4] [test] [client] $ redis-cli -p 6380 GET foo
[tester::#YG4] Received: "" (no content received)
[tester::#YG4]            ^ error
[tester::#YG4] Error: Expected start of a new RESP2 value (either +, -, :, $ or *)
[tester::#YG4] Test failed

Here’s my code:

Hey @yinkam, when your Redis server runs as a replica, the connection to the master needs to be set to non-blocking mode after the handshake completes.

Let me know if you’d like any further clarification!

Ohh that’s really helpful thanks! I set it to turn off non-blocking before the handshake, and I completely forgot about this.

Hi @andy1li, the fix helped with the “connection refused” error, but I’m still seeing the “no content received” error sporadically. I moved on to GETACK and WAIT commands, and sometimes my tests for #YD3 fail sporadically as well.

I’ve had to re-test and re-submit three or more times per stage to pass a stage. If this affects multiple users, it could significantly impact the learning experience.

I think I should identify the root cause of this issue, as I’m uncertain whether it is environmental or a code issue.

Failure Logs for #YD3

[tester::#YD3] Running tests for Stage #YD3 (Replication - ACKs with commands)
[tester::#YD3] Master is running on port 6379
[tester::#YD3] $ ./your_program.sh --port 6380 --replicaof “localhost 6379”
[your_program] Logs from your program will appear here!
[tester::#YD3] [handshake] [master] Waiting for replica to initiate handshake with “PING” command
[tester::#YD3] [handshake] [master] Received bytes: “*1\r\n$4\r\nPING\r\n”
[tester::#YD3] [handshake] [master] Received RESP array: [“PING”]
[tester::#YD3] [handshake] [master] ✔︎ Received [“PING”]
[tester::#YD3] [handshake] [master] Sent “PONG”
[tester::#YD3] [handshake] [master] Sent bytes: “+PONG\r\n”
[tester::#YD3] [handshake] [master] Waiting for replica to send “REPLCONF listening-port 6380” command
[tester::#YD3] [handshake] [master] Received bytes: “*3\r\n$8\r\nREPLCONF\r\n$14\r\nlistening-port\r\n$4\r\n6380\r\n”
[tester::#YD3] [handshake] [master] Received RESP array: [
[tester::#YD3] [handshake] [master] “REPLCONF”,
[tester::#YD3] [handshake] [master] “listening-port”,
[tester::#YD3] [handshake] [master] “6380”
[tester::#YD3] [handshake] [master] ]
[tester::#YD3] [handshake] [master] ✔︎ Received [
[tester::#YD3] [handshake] [master] “REPLCONF”,
[tester::#YD3] [handshake] [master] “listening-port”,
[tester::#YD3] [handshake] [master] “6380”
[tester::#YD3] [handshake] [master] ]
[tester::#YD3] [handshake] [master] Sent “OK”
[tester::#YD3] [handshake] [master] Sent bytes: “+OK\r\n”
[tester::#YD3] [handshake] [master] Waiting for replica to send “REPLCONF capa” command
[tester::#YD3] [handshake] [master] Received bytes: “*3\r\n$8\r\nREPLCONF\r\n$4\r\ncapa\r\n$6\r\npsync2\r\n”
[tester::#YD3] [handshake] [master] Received RESP array: [“REPLCONF”, “capa”, “psync2”]
[tester::#YD3] [handshake] [master] ✔︎ Received [“REPLCONF”, “capa”, “psync2”]
[tester::#YD3] [handshake] [master] Sent “OK”
[tester::#YD3] [handshake] [master] Sent bytes: “+OK\r\n”
[tester::#YD3] [handshake] [master] Waiting for replica to send “PSYNC” command
[tester::#YD3] [handshake] [master] Received bytes: “*3\r\n$5\r\nPSYNC\r\n$1\r\n?\r\n$2\r\n-1\r\n”
[tester::#YD3] [handshake] [master] Received RESP array: [“PSYNC”, “?”, “-1”]
[tester::#YD3] [handshake] [master] ✔︎ Received [“PSYNC”, “?”, “-1”]
[tester::#YD3] [handshake] [master] Sent “FULLRESYNC 75cd7bc10c49047e0d163660f3b90625b1af31dc 0”
[tester::#YD3] [handshake] [master] Sent bytes: “+FULLRESYNC 75cd7bc10c49047e0d163660f3b90625b1af31dc 0\r\n”
[tester::#YD3] [handshake] [master] Sending RDB file…
[tester::#YD3] [handshake] [master] Sent bytes: “$88\r\nREDIS0011\xfa\tredis-ver\x057.2.0\xfa\nredis-bits\xc0@\xfa\x05ctime\xc2m\b\xbce\xfa\bused-mem°\xc4\x10\x00\xfa\baof-base\xc0\x00\xff\xf0n;\xfe\xc0\xffZ\xa2”
[tester::#YD3] [handshake] [master] Sent RDB file.
[tester::#YD3] [test] [master] > REPLCONF GETACK *
[tester::#YD3] [test] [master] Sent bytes: “3\r\n$8\r\nREPLCONF\r\n$6\r\nGETACK\r\n$1\r\n\r\n”
[tester::#YD3] [test] [master] Received bytes: “*3\r\n$8\r\nREPLCONF\r\n$3\r\nACK\r\n$1\r\n0\r\n”
[tester::#YD3] [test] [master] Received RESP array: [“REPLCONF”, “ACK”, “0”]
[tester::#YD3] [test] [master] ✔︎ Received [“REPLCONF”, “ACK”, “0”]
[tester::#YD3] [propagation] [master] > PING
[tester::#YD3] [propagation] [master] Sent bytes: “*1\r\n$4\r\nPING\r\n”
[tester::#YD3] [test] [master] > REPLCONF GETACK *
[tester::#YD3] [test] [master] Sent bytes: “3\r\n$8\r\nREPLCONF\r\n$6\r\nGETACK\r\n$1\r\n\r\n”
[tester::#YD3] [test] [master] Received bytes: “*3\r\n$8\r\nREPLCONF\r\n$3\r\nACK\r\n$2\r\n37\r\n”
[tester::#YD3] [test] [master] Received RESP array: [“REPLCONF”, “ACK”, “37”]
[tester::#YD3] Expected argument #2 to be “51”, got “37”
[tester::#YD3] Test failed
[tester::#YD3] Terminating program
[tester::#YD3] Program terminated successfully

@gimmy1 Note that a single read from a TCP connection isn’t guaranteed to contain exactly one command. Your replica needs to be able to handle multiple commands arriving in the same read.

If you add a debug log like this:

You can verify that the read intended for the RDB transfer may also include additional data beyond the RDB payload itself.

Let me know if you’d like further clarification!