Replication Stage #ZN8: Replica Connection Array Empties Unexpectedly in Redis

Sraiti · May 11, 2024, 3:56pm

hello everyone

I’m stuck on replication:11

I’ve tried gathering the replicas connection in an array so i can propagate the write commands to them and they seems to gather normally until i receive a set command from the client i find out that the replicas array is empty so i fail at propagating the commands .

I would be extremely grateful if anyone could help me understand why the replica connection array becomes empty.

Here are my logs:

 [replication-11] Running tests for Replication > Stage #11: Single-replica propagation
remote: [replication-11] $ ./spawn_redis_server.sh --port 6379
remote: [your_program] Logs from your program will appear here!
remote: [your_program] {
remote: [your_program]   args: [ "/usr/local/bin/bun", "/app/app/main.ts", "--port", "6379" ],
remote: [your_program]   replicaOf: false,
remote: [your_program] }
remote: [your_program] MASTER
remote: [replication-11] replica: $ redis-cli PING
remote: [replication-11] replica: Sent bytes: "*1\r\n$4\r\nPING\r\n"
remote: [your_program] Master initialize RedisConnectionHandler {
remote: [your_program]   localport: 6379,
remote: [your_program]   remotePort: 6379,
remote: [your_program] }
remote: [your_program] Master initialize RedisConnectionHandler {
remote: [your_program]   localport: 6379,
remote: [your_program]   remotePort: 6379,
remote: [your_program] }
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "ping",
remote: [your_program] }
remote: [replication-11] replica: Received bytes: "+PONG\r\n"
remote: [replication-11] replica: Received RESP value: "PONG"
remote: [replication-11] Received "PONG"
remote: [replication-11] replica: $ redis-cli REPLCONF listening-port 6380
remote: [replication-11] replica: Sent bytes: "*3\r\n$8\r\nREPLCONF\r\n$14\r\nlistening-port\r\n$4\r\n6380\r\n"
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "replconf",
remote: [your_program] }
remote: [your_program] handleReplicaOf [ "REPLCONF", "listening-port", "6380" ]
remote: [your_program] {
remote: [your_program]   connectedReplicas: [ 6379 ],
remote: [your_program] }
remote: [replication-11] replica: Received bytes: "+OK\r\n"
remote: [replication-11] replica: Received RESP value: "OK"
remote: [replication-11] Received "OK"
remote: [replication-11] replica: $ redis-cli REPLCONF capa psync2
remote: [replication-11] replica: Sent bytes: "*3\r\n$8\r\nREPLCONF\r\n$4\r\ncapa\r\n$6\r\npsync2\r\n"
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [ 6379 ],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "replconf",
remote: [your_program] }
remote: [your_program] handleReplicaOf [ "REPLCONF", "capa", "psync2" ]
remote: [replication-11] replica: Received bytes: "+OK\r\n"
remote: [replication-11] replica: Received RESP value: "OK"
remote: [replication-11] Received "OK"
remote: [replication-11] replica: $ redis-cli PSYNC ? -1
remote: [replication-11] replica: Sent bytes: "*3\r\n$5\r\nPSYNC\r\n$1\r\n?\r\n$2\r\n-1\r\n"
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [ 6379 ],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "psync",
remote: [your_program] }
remote: [your_program] handling PSYNC
remote: [your_program] sending master replication id and the offset to the slave
remote: [your_program] start getting RDB FILE 
remote: [your_program] sending the RDB file to the slave {
remote: [your_program]   sentRdbFilesRecords: [ "8371b4fb1155b71f4a04d3e1bc3e18c4a990aeeb" ],
remote: [your_program] }
remote: [replication-11] replica: Received bytes: "+FULLRESYNC 8371b4fb1155b71f4a04d3e1bc3e18c4a990aeeb 0\r\n"
remote: [replication-11] replica: Received RESP value: "FULLRESYNC 8371b4fb1155b71f4a04d3e1bc3e18c4a990aeeb 0"
remote: [replication-11] Received "FULLRESYNC 8371b4fb1155b71f4a04d3e1bc3e18c4a990aeeb 0"
remote: [replication-11] Reading RDB file...
remote: [replication-11] replica: Received bytes: "$88\r\nREDIS0011\xfa\tredis-ver\x057.2.0\xfa\nredis-bits\xc0@\xfa\x05ctime\xc2m\b\xbce\xfa\bused-mem°\xc4\x10\x00\xfa\baof-base\xc0\x00\xff\xf0n;\xfe\xc0\xffZ\xa2"
remote: [replication-11] Received RDB file
remote: [replication-11] client: $ redis-cli SET foo 123
remote: [replication-11] client: Sent bytes: "*3\r\n$3\r\nSET\r\n$3\r\nfoo\r\n$3\r\n123\r\n"
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "set",
remote: [your_program] }
remote: [your_program] start set command :
remote: [replication-11] client: Received bytes: "+OK\r\n"
remote: [replication-11] client: Received RESP value: "OK"
remote: [replication-11] Received "OK"
remote: [replication-11] client: $ redis-cli SET bar 456
remote: [replication-11] client: Sent bytes: "*3\r\n$3\r\nSET\r\n$3\r\nbar\r\n$3\r\n456\r\n"
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "set",
remote: [your_program] }
remote: [your_program] start set command :
remote: [replication-11] client: Received bytes: "+OK\r\n"
remote: [replication-11] client: Received RESP value: "OK"
remote: [replication-11] Received "OK"
remote: [replication-11] client: $ redis-cli SET baz 789
remote: [replication-11] client: Sent bytes: "*3\r\n$3\r\nSET\r\n$3\r\nbaz\r\n$3\r\n789\r\n"
remote: [your_program] 6379 : connected Replicas {
remote: [your_program]   connectedReplicas: [],
remote: [your_program] }
remote: [your_program] {
remote: [your_program]   command: "set",
remote: [your_program] }
remote: [your_program] start set command :
remote: [replication-11] client: Received bytes: "+OK\r\n"
remote: [replication-11] client: Received RESP value: "OK"
remote: [replication-11] Received "OK"
remote: [replication-11] Sent 3 SET commands to master successfully.
remote: [replication-11] replica: Expecting SET foo 123 to be propagated
remote: [replication-11] Received: "" (no content received)
remote: [replication-11]            ^ error
remote: [replication-11] Error: Expected start of a new RESP2 value (either +, -, :, $ or *)
remote: [replication-11] Test failed
remote: [replication-11] Terminating program
remote: [replication-11] Program terminated successfully

And here's a snippet of my code where i gather the connections : (typescript)

 private handleReplicaOf(parsedRequest: string[]) {
    console.log("handleReplicaOf", parsedRequest);

    if (parsedRequest.includes("listening-port")) {
      const replicaPort = parsedRequest.pop();

      if (replicaPort) {
        this.replicas.push(this.connection);
        console.log({
          connectedReplicas: this.replicas.map((r) => r.remotePort),
        });
      }
    }

    this.writeResponse(`+OK${this.CRLF}`);
  }

here is the full code in Github :

github.com

Sraiti/codecrafters-redis-typescript/blob/master/app/redisConnectionHandler.ts

import { Roles, mapToString, redisProtocolParser } from "./helpers.ts";
import * as net from "node:net";

import { Buffer } from "node:buffer";
import { mapStore, replicas } from "./replicasManager.ts";
import { RedisReplicationClient } from "./redisReplicationHandler.ts";

enum Commands {
  PING = "PING",
  ECHO = "ECHO",
  SET = "SET",
  GET = "GET",
  DELETE = "DEL",
  INFO = "INFO",
  REPLCONF = "REPLCONF",
  PSYNC = "PSYNC",
}
class RedisConnectionHandler {
  // private mapStore = new Map<
  //   string,

This file has been truncated. show original

Any insights or suggestions would be immensely appreciated. Thank you in advance for your time and help!

rohitpaulk · May 13, 2024, 12:10pm

@Sraiti Where is the initial value for this.replicas (empty array) set? Is it possible that logic is running again at some point?

Sraiti · May 13, 2024, 9:56pm

Thank you for the reply.

To provide more context, this.replicas is a field in my RedisConnectionHandler class. It is initialized as an empty array and is intended to store the connections established during the replication handshake process, allowing me to propagate commands to replicas.

Sraiti · May 15, 2024, 11:08pm

Hello everyone,

I’ve resolved the issue I encountered with the replica connection array emptying during command propagation. The problem was due to the array’s scope being limited to individual connections within the class.

or at least that what i think caused the problem .

Solution: I moved the replica array to a separate file to manage it as a global state. This ensures that the array maintains its state across all connections.

// sharedState.ts
import * as net from "node:net";

export const replicas: net.Socket[] = [];

This change keeps the connection array persistent and shared across the server, fixing the issue with ‘SET’ command propagation. Hope this helps anyone facing similar challenges!

you can find the new code in my repo : GitHub - Sraiti/codecrafters-redis-typescript: Building Redis

Best regards,

system · May 20, 2024, 11:08pm

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.

system · June 4, 2024, 2:03am

Note: I’ve updated the title of this post to include the stage ID (#ZN8). You can learn about the stages rename here: Upcoming change: Stages overhaul.

Topic		Replies	Views
Replication Stage #ZN8 Help Challenges challenge:redis	4	168	June 4, 2024
SOLVED: issue with Single-replica propagation stage #ZN8 Challenges challenge:redis	6	330	June 4, 2024
Stuck on command replication #ZN8 tests Challenges challenge:redis	11	202	June 4, 2024
Replication #ZN8: Feedback with more detail on problem? Challenges challenge:redis	9	242	June 4, 2024
Replication Stage (#XV6) Slave receive nothing on occasion Challenges challenge:redis	4	187	June 4, 2024

Replication Stage #ZN8: Replica Connection Array Empties Unexpectedly in Redis

Related topics