What is the purpose of the Kafka challenge?

Hello,

I discovered codecrafters by typing “Make your own kafka” in google, and this is my first challenge.

Seeing the name of the challenge, my expectations were to build step by step a storage system with an append-only log, handle partitioning, replication, maybe a basic consensus algo, etc.

I’m roughly in the middle of the challenge (#eg2), but so far the challenge was to parse bytes from a socket, and send a correct bytes answer back to client.

I learned a lot of stuff by doing the challenge, but not what I was expecting.

Thus my question, as most of code I wrote was bytes parsing/writing, why call it Make your own Kafka? I mean, we can also call it “Make your own s3”, the only thing to change would be the links to documentation, s3 instead of kafka, and update the expected bytes content in tests. Its an oversimplification of course, but do you see my point?

My question is genuine. Otherwise, I find your platform awesome, very easy and smooth to use. Thanks for your work on that.

Vlad

1 Like

Hey @VladislavSokolovKpler, I totally get where you’re coming from.

The Kafka challenge is still in development, and you’re seeing the early stages.

The initial stages focus on the protocol layer, which is essential for testing user implementations written in almost two dozen languages, and for compatibility + interop with official Kafka brokers.

More extensions are on the way. Stay tuned!

2 Likes

Hi @andy1li . Thank you so much for your reply! I really appreciate your guy’s hard work on creating these tutorials.

I have a quick question: I’m new to codecrafters. I’m also still pretty new to system-level programming, but I’m hoping to take on a comprehensive project that helps me deepen my understanding of networking. That is, I’m looking for a fully developed challenge with networking features. In your opinion, which tutorial would be the best fit for that goal?

Hi @andy1li ,
thanks for your answer, makes sense to me now!

1 Like

Hey @Andy2887, none of our challenges focus exclusively on networking, but many include meaningful networking components, and we’re actively adding new stages to all of them.

Build Your Own Redis is a solid choice. It currently has the most stages and offers plenty of hands-on experience with networking concepts.

1 Like

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.