Stuck in #EC3 with Elixir

[hippopotamus-709]

I managed to get to XJ7, but when i retest it i got error in the previous challenge

[tester::#EC3] Running tests for Stage #EC3 (Lists - Blocking retrieval)
[tester::#EC3] $ ./your_program.sh
[your_program] 09:30:10.777 [info] Server lifecycle event
[your_program] 09:30:10.787 [info] Server lifecycle event
[your_program] 09:30:10.790 [info] Server lifecycle event
[your_program] 09:30:10.795 [info] Server lifecycle event
[your_program] 09:30:10.795 [info] Server lifecycle event
[your_program] 09:30:10.795 [info] Business event
[tester::#EC3] [client-1] Connected (port 37120 → port 6379)
[tester::#EC3] [client-2] Connected (port 37124 → port 6379)
[tester::#EC3] [client-3] Connected (port 37126 → port 6379)
[tester::#EC3] [client-1] $ redis-cli BLPOP pineapple 0
[tester::#EC3] [client-1] Sent bytes: “*3\r\n$5\r\nBLPOP\r\n$9\r\npineapple\r\n$1\r\n0\r\n”
[tester::#EC3] [client-2] $ redis-cli BLPOP pineapple 0
[tester::#EC3] [client-2] Sent bytes: “*3\r\n$5\r\nBLPOP\r\n$9\r\npineapple\r\n$1\r\n0\r\n”
[tester::#EC3] [client-3] $ redis-cli RPUSH pineapple pear
[tester::#EC3] [client-3] Sent bytes: “*3\r\n$5\r\nRPUSH\r\n$9\r\npineapple\r\n$4\r\npear\r\n”
[your_program] No waiting clients
[your_program] Updated state after waking up clients: %{lists: %{“pineapple” => [“pear”]}, waiting_queues: %{}}
[tester::#EC3] [client-3] Received bytes: “:1\r\n”
[tester::#EC3] [client-3] Received RESP integer: 1
[tester::#EC3] [client-3] ︎ Received 1
[tester::#EC3] [client-2] Expecting no response
[tester::#EC3] [client-2] ︎ No response received yet
[tester::#EC3] [client-1] Expecting response of BLPOP command
[tester::#EC3] Received: “” (no content received)
[tester::#EC3] ^ error
[tester::#EC3] Error: Expected start of a new RESP2 value (either +, -, :, $ or *)
[tester::#EC3] Test failed
[tester::#EC3] Terminating program
[your_program] 09:30:12.894 [notice] SIGTERM received - shutting down
[your_program]
[your_program] 09:30:12.899 [info] Server lifecycle event
[your_program] 09:30:12.894 client_ip=unknown [info] Client connection event
[tester::#EC3] Program terminated successfully

I’m working on a Redis clone and implementing the BLPOP command using GenServer. However, I’m noticing that the results of BLPOP aren’t consistent when multiple clients block on the same list.

For example, when I run two redis-cli blpop orange 0 commands simultaneously, sometimes the first client receives the popped element first, and sometimes it’s the second client. The calls appear almost concurrent since there’s no deliberate delay (sleep) in the implementation.

When I test the behavior manually, responses seem to route correctly to the client that has been waiting the longest. However, the actual implementation behaves inconsistently.

Here is an example of the behavior:

redis-cli blpop orange 0 & redis-cli blpop orange 0 &
[1] 136144
[2] 136145

"orange"
"oke"
[1] - 136144 done redis-cli blpop orange 0

"orange"
"oke"
[2] + 136145 done redis-cli blpop orange 0

redis-cli blpop orange 0 & redis-cli blpop orange 0 &
[1] 136148
[2] 136149

"orange"
"oke"
[1] - 136148 done redis-cli blpop orange 0

Could this be due to how GenServer handles blocking clients, or am I missing something in the queuing or notification logic? Any pointers on ensuring BLPOP consistently returns the element to the longest-waiting client would be greatly appreciated.

Thanks in advance!

lib/command_processor/list_commands_server.ex

[andy1li]

Hey @hippopotamus-709, looks like there’s a race condition where waking_up_waiting_client is executed before register_waiting_client:

[hippopotamus-709]

Thank you for your reply. It seems like the register_waiting_client requires more time to process than waking_up_waiting_client .

I may need to implement “locking” mechanism just to make sure no waking_up is allowed if theres a registering waiting client is happening.

I wonder how the original redis code solve this or others submission solve it ?

Thanks

[andy1li]

The official Redis first checks if the keys exist, and blocks if they don’t:

src/t_list.c

src/t_list.c

The logic is quite similar to your approach, but its C data structures may be more efficient.

---

You can also check the Code Examples.

[andy1li]

@hippopotamus-709 A heads-up: We plan to stop enforcing the assertion order in this stage. That said, it may take a little time before the change is rolled out.

[hippopotamus-709]

Noted. Meanwhile, I will try to make it atomic. The current implementation handles checking waiting clients and registering a waiting client simultaneously, which may create a race window.

[hippopotamus-709]

I solved the issue by making it atomic, the problem was the process of registering waiting client happens on a separate gen server call causing a racing condition.

But i still have issue, EC3 is becoming intermittent. As you can see in the screenshot, the blpop request comes at the same time, so the order of the list could be vary. So if the plan to stop enforcing the assertion order is implemented, or the sequence of the request can be slightly different I think the problem will go away. It is incredibly tricky to handle this kind of situation