Second replica fails handshake (#HD5)

hlud6646 · January 15, 2025, 10:21pm

I’m stuck on Stage #(#HD5 Multi Replica Command Propagation).

Problem

This post follows on from https://forum.codecrafters.io/t/second-replica-fails-handshake-in-redis-challenge-hd5/3854. The problem, as in that thread, is that I am able to spin up multiple replicas locally, with

Terminal 1: ./your_program
Terminal 2: ./your_program --port 1234 --replicaof "localhost 6379"
Terminal 3: ./your_program --port 1235 --replicaof "localhsot 6379"

with successful handshakes and write requests to the master propagated to the slaves, but that the corresponding test stage fails.

Attempted Fix

In that post @andy1li suggested moving away from using BufferedReader and readLine. I have refactored the project, relying on the more primitive InputStream.read(). The original problem persists

Logs

From the master:

Bound to port 6379.
Waiting for connections.
	Accepted connection from Socket[addr=/127.0.0.1,port=54530,localport=6379]
		Received Ping [List()]
		Sending response to client at 54530.
		Received Replconf [List(listening-port, 1234)]
		Adding Socket[addr=/127.0.0.1,port=54530,localport=6379] to slaves.
		Sending response to client at 54530.
		Received Replconf [List(capa, psync2)]
		Sending response to client at 54530.
		Sending response to client at 54530.
			Sending RDB to slave: 54530
	Accepted connection from Socket[addr=/127.0.0.1,port=50850,localport=6379]
		Received Ping [List()]
		Sending response to client at 50850.
		Received Replconf [List(listening-port, 1235)]
		Adding Socket[addr=/127.0.0.1,port=50850,localport=6379] to slaves.
		Sending response to client at 50850.
		Received Replconf [List(capa, psync2)]
		Sending response to client at 50850.
		Sending response to client at 50850.
			Sending RDB to slave: 50850
	Accepted connection from Socket[addr=/127.0.0.1,port=33302,localport=6379]
		Received Set [List(asd, qwe)]
		Sending response to client at 33302.
		Propogating write request to 2 replicas.
			Propogating req to socket Socket[addr=/127.0.0.1,port=54530,localport=6379]
			Propogating req to socket Socket[addr=/127.0.0.1,port=50850,localport=6379]
		End of input stream. Closing socket.
	Accepted connection from Socket[addr=/127.0.0.1,port=43506,localport=6379]
		Received Set [List(foo, bar)]
		Sending response to client at 43506.
		Propogating write request to 2 replicas.
			Propogating req to socket Socket[addr=/127.0.0.1,port=54530,localport=6379]
			Propogating req to socket Socket[addr=/127.0.0.1,port=50850,localport=6379]
		End of input stream. Closing socket.

From the 1st slave:

Bound to port 1234.
[Handshake] Starting handshake.
[Handshake] Opened socket Socket[addr=localhost/127.0.0.1,port=6379,localport=54530]
[Handshake] Sending PING to 6379
[Handshake] Received PONG
[Handshake] Sending first replconf to 6379
[Handshake] Received OK
[Handshake] Sending second replconf to 6379
[Handshake] Received OK
[Handshake] Sending psync to 6379
[Handshake] Receiving Database...
[Handshake] Received 88 bytes of database.
[Handshake] Handshake complete!
Waiting for requests.
	Accepted connection from 6379 with acceptWriteRequests = true
		Received Set [List(asd, qwe)]
		Received Set [List(foo, bar)]

From the second slave:

Bound to port 1235.
[Handshake] Starting handshake.
[Handshake] Opened socket Socket[addr=localhost/127.0.0.1,port=6379,localport=50850]
[Handshake] Sending PING to 6379
[Handshake] Received PONG
[Handshake] Sending first replconf to 6379
[Handshake] Received OK
[Handshake] Sending second replconf to 6379
[Handshake] Received OK
[Handshake] Sending psync to 6379
[Handshake] Receiving Database...
[Handshake] Received 88 bytes of database.
[Handshake] Handshake complete!
Waiting for requests.
	Accepted connection from 6379 with acceptWriteRequests = true
		Received Set [List(asd, qwe)]
		Received Set [List(foo, bar)]

and finally the output of codecrafters test:

...(build steps)...

[tester::#HD5] Running tests for Stage #HD5 (Replication - Multi Replica Command Propagation)
[tester::#HD5] $ ./your_program.sh --port 6379
[your_program] Bound to port 6379.
[your_program] Waiting for connections.
[tester::#HD5] [handshake] Creating replica: 1
[tester::#HD5] [handshake] replica-1: $ redis-cli PING
[tester::#HD5] [handshake] replica-1: Sent bytes: "*1\r\n$4\r\nPING\r\n"
[your_program] 	Accepted connection from Socket[addr=/127.0.0.1,port=46370,localport=6379]
[your_program] 	Accepted connection from Socket[addr=/127.0.0.1,port=46374,localport=6379]
[your_program] 		Received Ping [List()]
[your_program] 		Sending response to client at 46374.
[tester::#HD5] [handshake] replica-1: Received bytes: "+PONG\r\n"
[tester::#HD5] [handshake] replica-1: Received RESP simple string: "PONG"
[tester::#HD5] [handshake] Received "PONG"
[tester::#HD5] [handshake] replica-1: > REPLCONF listening-port 6380
[tester::#HD5] [handshake] replica-1: Sent bytes: "*3\r\n$8\r\nREPLCONF\r\n$14\r\nlistening-port\r\n$4\r\n6380\r\n"
[your_program] 		Received Replconf [List(listening-port, 6380)]
[your_program] 		Adding Socket[addr=/127.0.0.1,port=46374,localport=6379] to slaves.
[your_program] 		Sending response to client at 46374.
[tester::#HD5] [handshake] replica-1: Received bytes: "+OK\r\n"
[tester::#HD5] [handshake] replica-1: Received RESP simple string: "OK"
[tester::#HD5] [handshake] Received "OK"
[tester::#HD5] [handshake] replica-1: > REPLCONF capa psync2
[tester::#HD5] [handshake] replica-1: Sent bytes: "*3\r\n$8\r\nREPLCONF\r\n$4\r\ncapa\r\n$6\r\npsync2\r\n"
[your_program] 		Received Replconf [List(capa, psync2)]
[tester::#HD5] [handshake] replica-1: Received bytes: "+OK\r\n"
[tester::#HD5] [handshake] replica-1: Received RESP simple string: "OK"
[your_program] 		Sending response to client at 46374.
[tester::#HD5] [handshake] Received "OK"
[tester::#HD5] [handshake] replica-1: > PSYNC ? -1
[tester::#HD5] [handshake] replica-1: Sent bytes: "*3\r\n$5\r\nPSYNC\r\n$1\r\n?\r\n$2\r\n-1\r\n"
[your_program] 		Sending response to client at 46374.
[your_program] 			Sending RDB to slave: 46374
[tester::#HD5] [handshake] replica-1: Received bytes: "+FULLRESYNC mgwcM0vHZba5zg7bDypkRWmIopxi4V5EBIJT8y75 0\r\n"
[tester::#HD5] [handshake] replica-1: Received RESP simple string: "FULLRESYNC mgwcM0vHZba5zg7bDypkRWmIopxi4V5EBIJT8y75 0"
[tester::#HD5] [handshake] Received "FULLRESYNC mgwcM0vHZba5zg7bDypkRWmIopxi4V5EBIJT8y75 0"
[tester::#HD5] [handshake] Reading RDB file...
[tester::#HD5] [handshake] replica-1: Received bytes: "$88\r\nREDIS0011\xfa\tredis-ver\x057.2.0\xfa\nredis-bits\xc0@\xfa\x05ctime\xc2m\b\xbce\xfa\bused-mem°\xc4\x10\x00\xfa\baof-base\xc0\x00\xff\xf0n;\xfe\xc0\xffZ\xa2"
[tester::#HD5] [handshake] Received RDB file
[tester::#HD5] Creating replica: 2
[tester::#HD5] [handshake] replica-2: $ redis-cli PING
[tester::#HD5] [handshake] replica-2: Sent bytes: "*1\r\n$4\r\nPING\r\n"
[tester::#HD5] Received: "" (no content received)
[tester::#HD5]            ^ error
[tester::#HD5] Error: Expected start of a new RESP2 value (either +, -, :, $ or *)
[tester::#HD5] Test failed
[tester::#HD5] Terminating program
[your_program] 		End of input stream. Closing socket.
[your_program] 	Accepted connection from Socket[addr=/127.0.0.1,port=46382,localport=6379]
[your_program] 		Received Ping [List()]
[your_program] 		Sending response to client at 46382.
[tester::#HD5] Program terminated successfully

Any help would be greatly appreciated!

andy1li · January 16, 2025, 4:09am

I’ll take a look shortly.

andy1li · January 16, 2025, 11:29am

Hi @hlud6646, I added some logs, and it seems that Future hangs when the second replica tries to send PING.

Future only continues to execute after our tester tried to terminate your program:

However, the issue can be resolved by using Thread :

val clientThread = new Thread(() => {
  handleClient(clientSocket, acceptWriteRequests = true)
})
clientThread.start()

But why?

It seems that the default size of the thread pool for Future is determined by the number of CPU cores:

However, the CPUs of our testers only have 2 cores, which makes them unable to handle a 3rd connection:

Client
Replica 1
Replica 2

hlud6646 · January 16, 2025, 8:51pm

Fantastic. Thank you Andy for another instant reply!

Serves me right for ignoring all the warnings about blindly importing the default execution context!

hlud6646 · January 17, 2025, 4:04am

One other thing @andy1li
How do you get that cool looking italic font for classes

andy1li · January 17, 2025, 11:55am

Oh, it’s part of the font I use. Here’re some fonts that support cursive italics:

https://www.reddit.com/r/ProgrammingFonts/comments/1foeqzt/cursive_italic_font/

system · January 22, 2025, 11:56am

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Second replica fails handshake in Redis challenge (#HD5) Challenges challenge:redis	5	36	January 10, 2025
Redis multiple replica fails with repeated command #HD5 Challenges challenge:redis	2	13	February 7, 2025
Replication Stage #YD3 Missing GEKACK Challenges challenge:redis	3	137	June 4, 2024
[Gleam] Stuck on Redis Challenge, Replication, Stage 13 Challenges challenge:redis	3	87	June 5, 2024
Can't send synchronization commands to Redis replica Challenges challenge:redis	3	69	June 5, 2024

Second replica fails handshake (#HD5)

Problem

Attempted Fix

Logs

Related topics