Need help with stage 9 DNS (parsing compressed questions)

Challenges
1

I’m stuck on Stage #YC9.

I have written code to parse the compressed question and given response accordingly. I am able to parse the question and response but when I run codecrafters test, it gives error saying: ‘buffer size too small.’

Here are my logs:

[tester::#YC9] Running tests for Stage #YC9 (Parse compressed packet)
[tester::#YC9] Starting DNS server on 127.0.0.1:2053
[tester::#YC9] Running program
[tester::#YC9] DNS resolver listening on 127.0.0.1:5354
[tester::#YC9] Connecting to 127.0.0.1:2053 using UDP
[your_program] 2024-11-17 20:37:35 | line: 12 | app.logger/main | INFO: Logs from your program will appear here!
[tester::#YC9] Querying the following in the same request (Messages with >> prefix are part of this log)
[tester::#YC9] >> ;abc.longassdomainname.com.   IN       A
[tester::#YC9] >> ;def.longassdomainname.com.   IN       A
[tester::#YC9] Sending Request: (Messages with >>> prefix are part of this log)
[tester::#YC9] >>> ;; opcode: QUERY, status: NOERROR, id: 33217
[tester::#YC9] >>> ;; flags: rd; QUERY: 2, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
[tester::#YC9] >>> 
[tester::#YC9] >>> ;; QUESTION SECTION:
[tester::#YC9] >>> ;abc.longassdomainname.com.  IN       A
[tester::#YC9] >>> ;def.longassdomainname.com.  IN       A
[tester::#YC9] >>> 
[your_program] 2024-11-17 20:37:36 | line: 22 | app.logger/main | DEBUG: incoming raw data: b'\xd0\xed\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x0ccodecrafters\x02io\x00\x00\x01\x00\x01'
[your_program] 2024-11-17 20:37:36 | line: 24 | app.logger/main | DEBUG: parsed header dict: (53485, 256, 1, 0, 0, 0)
[your_program] 2024-11-17 20:37:36 | line: 26 | app.logger/main | DEBUG: parsed questions: [{'qname': 'codecrafters.io', 'qtype': 1, 'qclass': 1}]
[your_program] 2024-11-17 20:37:36 | line: 28 | app.logger/main | DEBUG: parsed flags: {'qr': 0, 'opcode': 0, 'aa': 0, 'tc': 0, 'rd': 1, 'ra': 0, 'z': 0, 'rcode': 0}
[your_program] 2024-11-17 20:37:36 | line: 72 | app.logger/main | DEBUG: response header: {'id': 53485, 'qr': 1, 'opcode': 0, 'aa': 0, 'tc': 0, 'rd': 1, 'ra': 0, 'z': 0, 'rcode': 0, 'qdcount': 1, 'ancount': 1, 'nscount': 0, 'arcount': 0}
[your_program] 2024-11-17 20:37:36 | line: 73 | app.logger/main | DEBUG: response questions:  [{'qname': 'codecrafters.io', 'qtype': 1, 'qclass': 1}]
[your_program] 2024-11-17 20:37:36 | line: 74 | app.logger/main | DEBUG: response answers:  [{'name': 'codecrafters.io', 'type': 1, 'domain_class': 1, 'ttl': 60, 'rdlength': 4, 'rdata': '8.8.8.8'}]
[your_program] 2024-11-17 20:37:36 | line: 76 | app.logger/main | DEBUG: response bytes: b'\xd0\xed\x81\x00\x00\x01\x00\x01\x00\x00\x00\x00\x0ccodecrafters\x02io\x00\x00\x01\x00\x01\x0ccodecrafters\x02io\x00\x00\x01\x00\x01\x00\x00\x00<\x00\x048.8.8.8'
[your_program] 2024-11-17 20:37:36 | line: 22 | app.logger/main | DEBUG: incoming raw data: b'\x81\xc1\x01\x00\x00\x02\x00\x00\x00\x00\x00\x00\x03abc\x11longassdomainname\x03com\x00\x00\x01\x00\x01\x03def\xc0\x10\x00\x01\x00\x01'
[your_program] 2024-11-17 20:37:36 | line: 24 | app.logger/main | DEBUG: parsed header dict: (33217, 256, 2, 0, 0, 0)
[your_program] 2024-11-17 20:37:36 | line: 26 | app.logger/main | DEBUG: parsed questions: [{'qname': 'abc.longassdomainname.com', 'qtype': 1, 'qclass': 1}, {'qname': 'def.longassdomainname', 'qtype': 1, 'qclass': 1}]
[your_program] 2024-11-17 20:37:36 | line: 28 | app.logger/main | DEBUG: parsed flags: {'qr': 0, 'opcode': 0, 'aa': 0, 'tc': 0, 'rd': 1, 'ra': 0, 'z': 0, 'rcode': 0}
[your_program] 2024-11-17 20:37:36 | line: 72 | app.logger/main | DEBUG: response header: {'id': 33217, 'qr': 1, 'opcode': 0, 'aa': 0, 'tc': 0, 'rd': 1, 'ra': 0, 'z': 0, 'rcode': 0, 'qdcount': 2, 'ancount': 2, 'nscount': 0, 'arcount': 0}
[your_program] 2024-11-17 20:37:36 | line: 73 | app.logger/main | DEBUG: response questions:  [{'qname': 'abc.longassdomainname.com', 'qtype': 1, 'qclass': 1}, {'qname': 'def.longassdomainname', 'qtype': 1, 'qclass': 1}]
[tester::#YC9] DNS query failed: dns: buffer size too small.
[tester::#YC9] If you are seeing this after a while then it is likely that your server is not responding with appropriate id
[tester::#YC9] Test failed
[tester::#YC9] Terminating program
[tester::#YC9] Shutting down DNS resolver server...
[your_program] 2024-11-17 20:37:36 | line: 74 | app.logger/main | DEBUG: response answers:  [{'name': 'abc.longassdomainname.com', 'type': 1, 'domain_class': 1, 'ttl': 60, 'rdlength': 4, 'rdata': '8.8.8.8'}, {'name': 'def.longassdomainname', 'type': 1, 'domain_class': 1, 'ttl': 60, 'rdlength': 4, 'rdata': '9.9.9.9'}]
[your_program] 2024-11-17 20:37:36 | line: 76 | app.logger/main | DEBUG: response bytes: b'\x81\xc1\x81\x00\x00\x02\x00\x02\x00\x00\x00\x00\x03abc\x11longassdomainname\x03com\x00\x00\x01\x00\x01\x03def\x11longassdomainname\x00\x00\x01\x00\x01\x03abc\x11longassdomainname\x03com\x00\x00\x01\x00\x01\x00\x00\x00<\x00\x048.8.8.8\x03def\x11longassdomainname\x00\x00\x01\x00\x01\x00\x00\x00<\x00\x049.9.9.9'
[tester::#YC9] Program terminated successfully

And here’s a snippet of my code (question parser):

import struct


class Question:
    def __init__(self, qname, qtype, qclass):
        self.qname: str = qname
        self.qtype: int = qtype
        self.qclass: int = qclass

    def to_bytes(self):
        return self.__encode_qname(self.qname) + struct.pack(
            ">HH", self.qtype, self.qclass
        )

    def __encode_qname(self, qname):
        parts = qname.split(".")
        result = b""
        for part in parts:
            result += struct.pack("B", len(part)) + part.encode("ascii")
        return result + b"\x00"

    def __parse_qname(self, data, start, parsed_qname={}):
        """
        I will parse the byte data and retur the qname with the next start position
        """
        parts = []
        while True:
            length = data[start]
            if length == 0:
                break
            if (0xC0 & length) == 0xC0:
                pointer = ((length & 0x3F) << 8) | data[start + 1]
                if parsed_qname.get(pointer, None):
                    parts.append(parsed_qname[pointer])
                else:
                    qname, _ = self.__parse_qname(Question, data, pointer, parsed_qname)
                    parts.append(qname)
                start += 1
                break
            label = data[start + 1 : start + 1 + length]
            parts.append(data[start + 1 : start + 1 + length].decode("ascii"))
            parsed_qname[start] = label.decode("ascii")
            start += 1 + length
        return ".".join(parts), start + 1, parsed_qname

    def __parse_questions(self, data, start=12, parsed_qname={}, questions=[]):
        if start >= len(data):
            return questions

        qname, start, new_parsed_qname = self.__parse_qname(
            Question, data=data, start=start, parsed_qname=parsed_qname
        )
        qtype, qclass = struct.unpack(">HH", data[start : start + 4])
        questions.append(Question(qname, qtype, qclass))
        return self.__parse_questions(
            Question, data, start + 4, {**parsed_qname, **new_parsed_qname}, questions
        )

    @staticmethod
    def from_bytes(data):
        # qname, start = Question.__parse_qname(Question, data, 0)
        # qtype, qclass = struct.unpack(">HH", data[start : start + 4])
        # return (qname, qtype, qclass)
        return Question.__parse_questions(
            Question, data=data, parsed_qname={}, questions=[]
        )
2

Could you upload your code to GitHub and share the link? It will be much easier to debug if I can run it directly.

3

Sure, here is the link to the repo:

@andy1li

5

@andy1li This is solved now. I was not encoding the rdata properly.

But the problem is in previos steps test were passed. I think the test of answer section needs to be revisted by the codecrafters team.

6

Thanks for the feedback, and I’m glad to hear you resolved the issue with the rdata encoding!

Could you elaborate a bit on how we could improve the tests for the answer section? :handshake:

7

Sure, I tried sending dns query using dig command (without compression) and noticed a warning that said, “found 3 bytes extra in response”.

I also noticed that I was encoding “8.8.8.8” in the code but response in dig command was something else. So this led to identify something was wrong with rdata.

I was encoding the rdata as string.

self.rdata.encode("ascii")

I changed this to:

# modified_rdata = [8,8,8,8]
struct.pack("!BBBB", *modified_rdata)

In a nutshell, may be you can add a check on the response size in answer section if it’s not already there.

8

Thanks for the detailed explanation! I’ll keep you updated on any changes we make to the tester.