#XV6 Replica doesn't always receive commands from master

Challenges
1

With stage #XV6, the master’s messages (sending over the empty RDB file, then REPLCONF GETACK *) are received sometimes, but at other times, the replica doesn’t seem to receive anything. In the second case, my print statements aren’t being outputted to the terminal so I’m pretty confident the replica isn’t receiving anything.

This is despite the stage being completed, since the tests managed to work when pushing to GitHub at some point. Therefore, to reproduce this issue, you may need to try re-running the tests a few times to observe the problem.

Here are my logs:

remote: [tester::#YD3] Running tests for Stage #YD3 (Replication - ACKs with commands)
remote: [tester::#YD3] Master is running on port 6379
remote: [tester::#YD3] $ ./your_program.sh --port 6380 --replicaof "localhost 6379"
remote: [your_program] Logs from your program will appear here!
remote: [your_program] I'm a replica
remote: [tester::#YD3] [handshake] [master] Waiting for replica to initiate handshake with "PING" command
remote: [tester::#YD3] [handshake] [master] Received bytes: "*1\r\n$4\r\nPING\r\n"
remote: [tester::#YD3] [handshake] [master] Received RESP array: ["PING"]
remote: [tester::#YD3] [handshake] [master] ✔︎ Received ["PING"]
remote: [tester::#YD3] [handshake] [master] Sent "PONG"
remote: [tester::#YD3] [handshake] [master] Sent bytes: "+PONG\r\n"
remote: [tester::#YD3] [handshake] [master] Waiting for replica to send "REPLCONF listening-port 6380" command
remote: [tester::#YD3] [handshake] [master] Received bytes: "*3\r\n$8\r\nREPLCONF\r\n$14\r\nlistening-port\r\n$4\r\n6380\r\n"
remote: [tester::#YD3] [handshake] [master] Received RESP array: [
remote: [tester::#YD3] [handshake] [master]   "REPLCONF",
remote: [tester::#YD3] [handshake] [master]   "listening-port",
remote: [tester::#YD3] [handshake] [master]   "6380"
remote: [tester::#YD3] [handshake] [master] ]
remote: [tester::#YD3] [handshake] [master] ✔︎ Received [
remote: [tester::#YD3] [handshake] [master]   "REPLCONF",
remote: [tester::#YD3] [handshake] [master]   "listening-port",
remote: [tester::#YD3] [handshake] [master]   "6380"
remote: [tester::#YD3] [handshake] [master] ]
remote: [tester::#YD3] [handshake] [master] Sent "OK"
remote: [tester::#YD3] [handshake] [master] Sent bytes: "+OK\r\n"
remote: [tester::#YD3] [handshake] [master] Waiting for replica to send "REPLCONF capa" command
remote: [tester::#YD3] [handshake] [master] Received bytes: "*3\r\n$8\r\nREPLCONF\r\n$4\r\ncapa\r\n$6\r\npsync2\r\n"
remote: [tester::#YD3] [handshake] [master] Received RESP array: ["REPLCONF", "capa", "psync2"]
remote: [tester::#YD3] [handshake] [master] ✔︎ Received ["REPLCONF", "capa", "psync2"]
remote: [tester::#YD3] [handshake] [master] Sent "OK"
remote: [tester::#YD3] [handshake] [master] Sent bytes: "+OK\r\n"
remote: [tester::#YD3] [handshake] [master] Waiting for replica to send "PSYNC" command
remote: [tester::#YD3] [handshake] [master] Received bytes: "*3\r\n$5\r\nPSYNC\r\n$1\r\n?\r\n$2\r\n-1\r\n"
remote: [tester::#YD3] [handshake] [master] Received RESP array: ["PSYNC", "?", "-1"]
remote: [tester::#YD3] [handshake] [master] ✔︎ Received ["PSYNC", "?", "-1"]
remote: [tester::#YD3] [handshake] [master] Sent "FULLRESYNC 75cd7bc10c49047e0d163660f3b90625b1af31dc 0"
remote: [tester::#YD3] [handshake] [master] Sent bytes: "+FULLRESYNC 75cd7bc10c49047e0d163660f3b90625b1af31dc 0\r\n"
remote: [tester::#YD3] [handshake] [master] Sending RDB file...
remote: [tester::#YD3] [handshake] [master] Sent bytes: "$88\r\nREDIS0011\xfa\tredis-ver\x057.2.0\xfa\nredis-bits\xc0@\xfa\x05ctime\xc2m\b\xbce\xfa\bused-mem°\xc4\x10\x00\xfa\baof-base\xc0\x00\xff\xf0n;\xfe\xc0\xffZ\xa2"
remote: [tester::#YD3] [handshake] [master] Sent RDB file.
remote: [tester::#YD3] [test] [master] > REPLCONF GETACK *
remote: [tester::#YD3] [test] [master] Sent bytes: "*3\r\n$8\r\nREPLCONF\r\n$6\r\nGETACK\r\n$1\r\n*\r\n"
remote: [tester::#YD3] Received: "" (no content received)
remote: [tester::#YD3]            ^ error
remote: [tester::#YD3] Error: Expected start of a new RESP2 value (either +, -, :, $ or *)
remote: [tester::#YD3] Test failed
remote: [tester::#YD3] Terminating program
remote: [your_program] EOF
remote: [your_program] Problem: error reading from master connection
remote: [tester::#YD3] Program terminated successfully

And here’s a snippet of my code:

func handshakeMaster(host, port string) error {
	conn, err := net.Dial("tcp", net.JoinHostPort(host, port))
	if err != nil {
		return err
	}
	// defer conn.Close()

	readBuffer := make([]byte, 1024)

	// send PING to master
	conn.Write(encodeBulkArray([]string{"PING"}))
	conn.Read(readBuffer)

	// send REPLCONF twice to the master
	conn.Write(encodeBulkArray([]string{"REPLCONF", "listening-port", configRepl["port"]}))
	conn.Read(readBuffer)
	conn.Write(encodeBulkArray([]string{"REPLCONF", "capa", "psync2"}))
	conn.Read(readBuffer)

	// send PSYNC to the master
	conn.Write(encodeBulkArray([]string{"PSYNC", "?", "-1"}))
	conn.Read(readBuffer)

	go func() {
		defer conn.Close()
		masterReadBuffer := make([]byte, 1024)
		for {
			n, err := conn.Read(masterReadBuffer)
			if err != nil {
				fmt.Println(err)
				fmt.Println("Problem: error reading from master connection")
				return
			}

			if n == 0 {
				break
			}

			fmt.Println(string(masterReadBuffer[:n]))
			masterParsedArray, types, err := parseRESP(masterReadBuffer[:n])
			if err != nil {
				fmt.Println("Problem: error thrown when parsing RESP content from master")
				continue
			}

			for i, arr := range masterParsedArray {
				if len(arr) == 1 && types[i] == 0 {
					writeRDBFile([]byte(arr[0]))
				} else {
					if arr[0] == "SET" {
						handleSet(arr) // no OK response back to master
					} else if sliceEquals(arr, []string{"REPLCONF", "GETACK", "*"}) {
						conn.Write(encodeBulkArray([]string{"REPLCONF", "ACK", "0"}))
					}
				}
			}
		}
	}()

	return nil
}
3

I can’t say if it’s the same cause, but I had the same intermittent failure pattern. In my case, it was because I was assuming that when the empty RDB file came in, it would be at the head of the stream. But sometimes (apparently not always), I would receive the FULLRESYNC and the empty RDB at once. I had to change my input handling to deal with the empty RDB being in the middle.

2 Likes
5

I think its the same thing. Thanks, I should be able to work it out now.

7

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.