# Unable to read the metalog file in multipe fetch messages stage of kafka

**URL:** <https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654>\
**Category:** Challenges\
**Tags:** challenge:kafka\
**Created:** [August 10, 2025, 5:44pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654 "2025-08-10T17:44:18Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![faith-in-het](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/faith-in-het/32/8759_2.png) [@faith-in-het](https://forum.codecrafters.io/u/faith-in-het)\
**Post date:** [August 10, 2025, 5:44pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/1 "2025-08-10T17:44:18Z")

</div>

Error reading log file: Error: Log file not found: /tmp/kraft-combined-logs/pax-0.log  
remote: [your\_program] at readFromFileBuffer (file:///app/app/fetch\_api\_request.js:106:19)  
remote: [your\_program] at file:///app/app/fetch\_api\_request.js:58:15  
remote: [your\_program] at Array.map ()  
remote: [your\_program] at handleFetchApiRequest (file:///app/app/fetch\_api\_request.js:24:8)  
remote: [your\_program] at Socket. (file:///app/app/main.js:28:7)  
remote: [your\_program] at Socket.emit (node:events:519:28)  
remote: [your\_program] at addChunk (node:internal/streams/readable:559:12)  
remote: [your\_program] at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)  
remote: [your\_program] at Readable.push (node:internal/streams/readable:390:5)  
remote: [your\_program] at TCP.onStreamRead (node:internal/stream\_base\_commons:190:23)  
remote: [your\_program] recordBatch { messages: , nextOffset: 0 }  
remote: [your\_program] node:buffer:587  
remote: [your\_program] throw new ERR\_INVALID\_ARG\_TYPE(  
remote: [your\_program] ^  
remote: [your\_program]  
remote: [your\_program] TypeError [ERR\_INVALID\_ARG\_TYPE]: The “list[8]” argument must be an instance of Buffer or Uint8Array. Received an instance of Object  
remote: [your\_program] at Function.concat (node:buffer:587:13)  
remote: [your\_program] at file:///app/app/fetch\_api\_request.js:60:45  
remote: [your\_program] at Array.map ()  
remote: [your\_program] at handleFetchApiRequest (file:///app/app/fetch\_api\_request.js:24:8)  
remote: [your\_program] at Socket. (file:///app/app/main.js:28:7)  
remote: [your\_program] at Socket.emit (node:events:519:28)  
remote: [your\_program] at addChunk (node:internal/streams/readable:559:12)  
remote: [your\_program] at readableAddChunkPushByteMode (node:internal/streams/readable:510:3)  
remote: [your\_program] at Readable.push (node:internal/streams/readable:390:5)  
remote: [your\_program] at TCP.onStreamRead (node:internal/stream\_base\_commons:190:23) {  
remote: [your\_program] code: ‘ERR\_INVALID\_ARG\_TYPE’  
remote: [your\_program] }  
remote: [your\_program]  
remote: [your\_program] Node.js v21.7.3  
remote: [tester::#FD8] EOF  
remote: [tester::#FD8] Test failed

unable to read the metadata file  
what could be the issue

> **[GitHub - faith-in-het/Build-your-own-Kafka](https://github.com/faith-in-het/Build-your-own-Kafka)**
>
> Contribute to faith-in-het/Build-your-own-Kafka development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [August 12, 2025, 9:13am UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/4 "2025-08-12T09:13:53Z")

</div>

Hey @faith-in-het, look like `quz-` is missing its partition suffix:

 ![image](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/2X/4/4f9672a02d1eaf8d267732f962d810d244652b74.jpeg)

Let me know if you’d like a hand digging into this further.

---

<div class="post-metadata">

**Author:** ![faith-in-het](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/faith-in-het/32/8759_2.png) [@faith-in-het](https://forum.codecrafters.io/u/faith-in-het)\
**Post date:** [August 12, 2025, 5:36pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/6 "2025-08-12T17:36:37Z")

</div>

i did some changes and got quz-0 but could not get quz-1 so yeah would like more help please

 ![image](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/2X/a/a40fa0cc3ed1821ef3b4d8f1f42c3e9fcdab12bd.png)

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [August 13, 2025, 12:26pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/8 "2025-08-13T12:26:47Z")

</div>

@faith-in-het Could you push your latest changes to GitHub?

The correct log file path should look like:

`/tmp/kraft-combined-logs/quz-0/00000000000000000000.log`

In your screenshot, it appears as:

`/tmp/kraft-combined-logs/quz-0.1og`

---

<div class="post-metadata">

**Author:** ![faith-in-het](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/faith-in-het/32/8759_2.png) [@faith-in-het](https://forum.codecrafters.io/u/faith-in-het)\
**Post date:** [August 15, 2025, 6:39am UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/12 "2025-08-15T06:39:42Z")

</div>

The thing is, when I push the code twice, one time it shows `quz-0` and the other time it shows `puz-0` for the same code. also you can check the code

---

<div class="post-metadata">

**Author:** ![faith-in-het](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/faith-in-het/32/8759_2.png) [@faith-in-het](https://forum.codecrafters.io/u/faith-in-het)\
**Post date:** [August 18, 2025, 4:55am UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/13 "2025-08-18T04:55:58Z")

</div>

I already pushed the latest code @andy1li

---

<div class="post-metadata">

**Author:** ![faith-in-het](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/faith-in-het/32/8759_2.png) [@faith-in-het](https://forum.codecrafters.io/u/faith-in-het)\
**Post date:** [August 21, 2025, 10:47am UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/14 "2025-08-21T10:47:18Z")

</div>

I already pushed the latest code @andy1li

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [August 28, 2025, 10:16pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/15 "2025-08-28T22:16:15Z")

</div>

Hey @faith-in-het, sorry for the delayed response!

Looks like the issue comes from using the buffer `partitionIndex` directly without parsing it first:

 ![image](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/4/f/4fb1502142bc9a231aa599436b660b771cd6b331.png)

 ![image](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/3/2/322cc697d59354fe1ed41ef3b945ec0093db6715.png)

---

<div class="post-metadata">

**Author:** ![faith-in-het](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/faith-in-het/32/8759_2.png) [@faith-in-het](https://forum.codecrafters.io/u/faith-in-het)\
**Post date:** [August 30, 2025, 6:19pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/17 "2025-08-30T18:19:04Z")

</div>

import fs from “fs”;

import { sendResponseMessage } from “./utils/index.js”;

export const handleFetchApiRequest = (connection, responseMessage, buffer) =\> {

const clientLength = buffer.subarray(12, 14);

const clientLengthValue = clientLength.readInt16BE();

const throttleTime = Buffer.from([0, 0, 0, 0]);

const errorCode = Buffer.from([0, 0]);

let responses = Buffer.from([1]);

const tagBuffer = Buffer.from([0]);

const sessionIdIndex = clientLengthValue + 28;

const sessionId = buffer.subarray(sessionIdIndex, sessionIdIndex + 4);

const \_sessionEpoch = buffer.subarray(sessionIdIndex + 4, sessionIdIndex + 8);

const topicArrayLength = buffer.subarray(

```
sessionIdIndex + 8,

sessionIdIndex + 9,

```

);

let topicIndex = sessionIdIndex + 9;

if (topicArrayLength.readInt8() \> 1) {

```
const topics = new Array(topicArrayLength.readInt8() - 1)

  .fill(0)

  .map((\_) => {

    const topicId = buffer.subarray(topicIndex, topicIndex + 16);

    topicIndex += 16;

    const logFile = fs.readFileSync(

      \`/tmp/kraft-combined-logs/\__cluster_metadata-0/00000000000000000000.log\`,

    );

    const logFileIndex = logFile.indexOf(topicId);

    let partitionError = Buffer.from(\[0, 100\]);

    let topicName = "";

    if (logFileIndex !== -1) {

      partitionError = Buffer.from(\[0, 0\]);

      topicName = logFile.subarray(logFileIndex - 3, logFileIndex);

    }

    const partitionArrayIndex = topicIndex;

    const partitionLength = buffer.subarray(

      partitionArrayIndex,

      partitionArrayIndex + 1,

    );

    const partitionIndex = buffer.subarray(

      partitionArrayIndex + 1,

      partitionArrayIndex + 5,

    );

    const highWaterMark = Buffer.from(new Array(8).fill(0));

    const last_stable_offset = Buffer.from(new Array(8).fill(0));

    const log_start_offset = Buffer.from(new Array(8).fill(0));

    const aborted_transactions = Buffer.from(\[0\]);

    const preferredReadReplica = Buffer.from(\[0, 0, 0, 0\]);

    

    // Get the record batch with proper compact format

    const recordBatch = logFileIndex === -1

      ? Buffer.from(\[0\]) // Just 0 for no records

      : readFromFileBuffer(topicName.toString(), partitionIndex.readInt32BE());

        

    console.log("recordBatch", recordBatch);

    const partitionArrayBuffer = Buffer.concat(\[

      partitionLength,

      partitionIndex,

      partitionError,

      highWaterMark,

      last_stable_offset,

      log_start_offset,

      aborted_transactions,

      preferredReadReplica,

      recordBatch,

      tagBuffer,

    \]);

    return Buffer.concat(\[topicId, partitionArrayBuffer, tagBuffer\]);

  });

responses = Buffer.concat(\[topicArrayLength, ...topics\]);

```

}

let fetchRequestResponse = {

```
correlationId: responseMessage.correlationId,

responseHeaderTagbuffer: tagBuffer,

throttleTime,

errorCode,

sessionId,

responses,

tagBuffer,

```

};

const messageSizeBuffer = Buffer.alloc(4);

messageSizeBuffer.writeInt32BE(

```
Buffer.concat(Object.values(fetchRequestResponse)).length

```

);

fetchRequestResponse = {

```
messageSize: messageSizeBuffer,

...fetchRequestResponse,

```

};

sendResponseMessage(connection, fetchRequestResponse);

};

function readFromFileBuffer(topicName, partitionIndex) {

```
try {

    const logFilePath = \`/tmp/kraft-combined-logs/${topicName}-${partitionIndex}/00000000000000000000.log\`;

    if (!fs.existsSync(logFilePath)) {

        console.log(\`Log file not found: ${logFilePath}\`);

        return Buffer.from(\[0\]); // No records

    }

    const logFile = fs.readFileSync(logFilePath);

    

    // For compact records format, we need to encode the length as a compact int

    // followed by the actual record data

    if (logFile.length === 0) {

        return Buffer.from(\[0\]); // No records

    }

    

    // Encode the length as a compact/varint and prepend it to the data

    const lengthBytes = encodeCompactInt(logFile.length);

    return Buffer.concat(\[lengthBytes, logFile\]);

    

} catch (error) {

    console.error("Error reading log file:", error);

    return Buffer.from(\[0\]); // No records

}

```

}

// Helper function to encode integers in compact/varint format

function encodeCompactInt(value) {

```
if (value === 0) {

    return Buffer.from(\[0\]);

}

const bytes = \[\];

let remaining = value;

while (remaining > 0x7F) {

    bytes.push((remaining & 0x7F) | 0x80);

    remaining >>>= 7;

}

bytes.push(remaining & 0x7F);

return Buffer.from(bytes);

```

}

i updated the fetch api code like this changed some format and added varint encoding thing so then it worked out for me  
Thank you so much for your support

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/7/0/700657133935c15703e22c7c8870394f6e6dc27d.svg) [@system](https://forum.codecrafters.io/u/system)\
**Post date:** [September 4, 2025, 6:19pm UTC](https://forum.codecrafters.io/t/unable-to-read-the-metalog-file-in-multipe-fetch-messages-stage-of-kafka/14654/18 "2025-09-04T18:19:37Z")

</div>

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.