Challenge stage #XV6: cannot figure out the problem (C++)

Challenges
1

Hello there,

I’ve been tackling the Redis challenge for the past few days, and I have been tearing my hair out because multiple stages seem to decide when they want to pass or not, and I have no idea why.

Here are the stages seemingly randomly causing fails:

  • ACKs with no commands
  • XREAD (blocking with $)
  • Command propagation in replicas
  • BLPOP (this one I know why, but am not sure how to handle)

If anyone could take a look at this, I would REALLY appreciate it! I will be publishing the code to GitHub so people can try and test it.

Here is the link: https://github.com/fortwoone/cpp_redis

Thanks in advance for answers, they would be greatly appreciated!

4

In an effort to fix command propagation, I deduced from the error and received commands that some commands were sent in one TCP segment, instead of one segment per command.
However, inconsistencies showed up later on (a test failing even when no change was made in the code it was targeting, most often) and it went downhill from there, and I don’t know why, even though I tried to figure it out by myself for a good while before seeking help here

5

Hey @fortwoone, great catch! As you correctly deduced, a single TCP read can include multiple commands:

image
image2630×406 68.2 KB

For instance, the PSYNC response and the RDB payload can arrive in the same read:

image
image1920×1080 182 KB

You’ll want to make sure your code can correctly handle such cases.

Let me know if you’d like any further clarification!

7

Thanks for the heads-up! Thing is, I did actually implement a workaround for this, but other stages seem to fail afterwards, and which one makes it fail seems to vary, and I can’t figure out why. I’ve been tearing my hair out on this for the past day

8

@fortwoone There might be multiple issues at play. Let’s tackle them one at a time.

From the screenshot, it looks like your code is attempting to read the RDB payload even when it has already been consumed:

Could you try addressing this part first?

I can’t figure out why

The reason is simply “a single TCP read can include multiple commands”. Once your code handles that case properly, things should start falling into place.

9

Well, honestly, I’m a bit stumped on this portion. If I comment out the RDB-consuming code, in some instances it works fine, but in others, it just ends up trying to interpret the RDB file as a command (which it isn’t) and throws an error (I did add this exception to ensure commands were actually parsed correctly)

11

@fortwoone Let’s take a step back and walk through it together.

Problem: A single TCP read can include multiple commands

Implication: We shouldn’t assume each TCP read maps to exactly one command. Namely, we should not hardcode the exact steps.

Solution: Making the request/response handling more flexible.

One approach is designing a general-purpose function that processes responses after every request. This function should contain a loop that handles an arbitrary number of commands.

12

Okay, so my attempt to resolve this was to memorise the last parsed command’s size so I could then advance from the starting position in the received buffer to parse the next one

13

At least that does seem to work

1 Like
14

But then other problems creep in and so I get confused. I don’t think this one is the root cause, though

15

@fortwoone Yes, there might be a few different things at play. What errors are you seeing now?

16

XREAD blocking with $ fails, and sometimes command propagation fails even though the code is identical

17

XREAD blocking with $ fails,

This might be a separate issue introduced by the recent fix.

and sometimes command propagation fails even though the code is identical

If it’s non-deterministic, it’s likely due to the same issue: multiple commands arriving in a single TCP read.

@fortwoone If you push your latest code, I’ll try taking another look tomorrow.

18

There