# SQLite retrieving data using index times out when testing

**URL:** <https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024>\
**Category:** Bug Reports\
**Tags:** challenge:sqlite\
**Created:** [July 26, 2025, 6:35pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024 "2025-07-26T18:35:35Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Qirall79](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/qirall79/32/14992_2.png) [@Qirall79](https://forum.codecrafters.io/u/Qirall79)\
**Post date:** [July 26, 2025, 6:35pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/1 "2025-07-26T18:35:35Z")

</div>

I’m stuck on Stage #nz8.

I’ve tried testing the test queries manually using the 1GB database, everything seems to be fast and not exceed 1 second (I know testing environment is different), but I keep getting 20s timeout when running `codecrafters test`

Here are my logs:

 ![Screenshot 2025-07-26 at 19.34.02](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/2X/2/2cf2e6adb1facad01d2ddb11df66e70051c6ba2a.png)

```auto
include relevant logs here (please make sure to keep the backticks around this!)

```

And here’s a snippet of my code for scanning the index pages and reading each rowId from table pages:

```python

void BTreeNode::readRowById(std::ifstream &file, Metadata& metadata, unsigned long pageNumber, Command& command, unsigned long id) {
    std::vector<unsigned short> offsets;
    unsigned long pageOffset = (pageNumber - 1) * metadata.pageSize;

    for (int i = 0; i < numberOfCells; i++)
    {
        offsets.push_back(Util::readShort(file));
    }

    for (auto offset : offsets)
    {
        file.seekg(pageOffset + offset);
        
        if (type == LEAF_NODE) {
            // payload size
            Util::readVarInt(file);
            
            // row id
            unsigned long rowId = Util::readVarInt(file);

            if (rowId != id)
                continue ;

            
            std::vector<unsigned long> serialTypes = parseRecordHeader(file);

            int i = 0;
            bool conditionFailed = false;
            std::map<std::string, std::string> values;

            for (auto serialType : serialTypes)
            {
                unsigned long columnLength;
                std::string columnType;

                if (serialType >= 13 && serialType % 2 == 1)
                {
                    columnLength = (serialType - 13) / 2;
                    columnType = "string";
                }
                else if (serialType >= 12 && serialType % 2 == 0)
                {
                    columnLength = (serialType - 12) / 2;
                    columnType = "blob";
                }
                else
                {
                    columnLength = serialType;
                    columnType = "int";

                    if (serialType == 5)
                        columnLength = 6;
                    if (serialType == 6)
                        columnLength = 8;
                }

                std::string value;
                if (columnType == "int")
                {
                    unsigned long val;

                    if (columnLength == 1)
                        val = Util::readByte(file);
                    else if (columnLength == 2)
                        val = Util::readShort(file);
                    else if (columnLength == 3)
                        val = Util::readMidInt(file);
                    else if (columnLength == 4)
                        val = Util::readInt(file);
                    else if (columnLength == 6)
                        val = Util::readMidLong(file);
                    else if (columnLength == 8)
                        val = Util::readLong(file);
                    else
                        val = 0;
                    value = std::to_string(val);
                }
                else
                {
                    value = Util::readString(file, columnLength);
                }

                values[metadata.columns[command.table][i]] = value;
                i++;
            }
            
            bool isFirst = true;

            values["id"] = std::to_string(rowId);

            if (!conditionFailed) {
                // display values
                for (auto column: command.columns) {
                    if (!isFirst)
                        std::cout << "|";
                    else
                        isFirst = false;
                    std::cout << values[column];
                }
                std::cout << std::endl;
            }
            return ;
        }
        else if (type == INTERIOR_NODE) {
            unsigned long leftChildPageNumber = Util::readInt(file);
            unsigned long key = Util::readVarInt(file);

            if (id < key) {
                BTreeNode childPage;
    
                file.seekg((leftChildPageNumber - 1) * metadata.pageSize);
                childPage.parseHeader(file, metadata);
                childPage.readRowById(file, metadata, leftChildPageNumber, command, id);
                return ;
            }
        }
    }

    if (type == INTERIOR_NODE) {
            BTreeNode childPage;

            file.seekg((rightMostPointer - 1) * metadata.pageSize);
            childPage.parseHeader(file, metadata);
            childPage.readRowById(file, metadata, rightMostPointer, command, id);
    }
}

std::vector<unsigned long> BTreeNode::scanIndex(std::ifstream &file, Metadata& metadata, unsigned long pageNumber, Command& command) {
    std::vector<unsigned short> offsets;
    std::vector<unsigned long> validIndexes;
    unsigned long pageOffset = (pageNumber - 1) * metadata.pageSize;

    for (int i = 0; i < numberOfCells; i++)
    {
        offsets.push_back(Util::readShort(file));
    }
    
    if (type == LEAF_INDEX_NODE) {
        for (auto offset : offsets)
        {
            file.seekg(pageOffset + offset);

            // payload size
            Util::readVarInt(file);
    
            std::vector<unsigned long> serialTypes = parseRecordHeader(file);
    
            std::vector<std::string> values = parseValues(file, serialTypes);
    
            if (values[0] == command.condition.second)
                validIndexes.push_back(std::stoul(values[1]));
        }
    }
    else if (type == INTERIOR_INDEX_NODE) {
        // use signed integers to avoid unsigned underflow
        int start = 0;
        int end = numberOfCells - 1;
        int mid;
        unsigned long targetChildPage = 0;
        
        // binary search for page to visit
        while (start <= end) {
            mid = (start + end) / 2;
            
            file.seekg(pageOffset + offsets[mid]);

            unsigned long childPageNumber = Util::readInt(file);
    
            Util::readVarInt(file);
    
            std::vector<unsigned long> serialTypes = parseRecordHeader(file);
    
            std::vector<std::string> values = parseValues(file, serialTypes);
            
            if (values[0] == command.condition.second) {
                validIndexes.push_back(std::stoul(values[1]));
                targetChildPage = childPageNumber;
                break;
            }
            else if (values[0] > command.condition.second) {
                targetChildPage = childPageNumber;
                end = mid - 1;
            } else {
                start = mid + 1;
            }
        }

        // no exact match found, check the rightmost pointer
        if (targetChildPage == 0 && rightMostPointer > 0) {
            targetChildPage = rightMostPointer;
        }

        if (targetChildPage > 0) {
            BTreeNode childPage;
    
            file.seekg((targetChildPage - 1) * metadata.pageSize);
            childPage.parseHeader(file, metadata);

            std::vector<unsigned long> childIndexes = childPage.scanIndex(file, metadata, targetChildPage, command);

            validIndexes.insert(validIndexes.end(), childIndexes.begin(), childIndexes.end());
        }
    }
    
    return validIndexes;
}

```

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [July 27, 2025, 12:13pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/4 "2025-07-27T12:13:55Z")

</div>

Hey @Qirall79, I tried running your code against the previous stages, but it’s actually no longer passing a previous stage **#RF3 (Filter data with a WHERE clause)**.

 ![image](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/2X/8/8464e7c981f6e98a8ba10e67b6a38b3f98154f00.png)

**Suggestions** :

1. Use [our CLI](https://docs.codecrafters.io/cli/installation) to test against previous stages by running:

```auto
codecrafters test --previous

```

1. Focus on fixing the early stages first, as later stages depend on them.

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [July 27, 2025, 12:20pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/5 "2025-07-27T12:20:43Z")

</div>

Looks like there is something wrong with our tester as well. Looking into it.

---

<div class="post-metadata">

**Author:** ![Qirall79](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/qirall79/32/14992_2.png) [@Qirall79](https://forum.codecrafters.io/u/Qirall79)\
**Post date:** [July 27, 2025, 4:57pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/6 "2025-07-27T16:57:56Z")

</div>

Hello, I just noticed I didn’t push my latest version. It passes the previoud stages now, but the timeout is still an issue for the last stage.

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [July 27, 2025, 5:20pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/7 "2025-07-27T17:20:23Z")

</div>

Yes, we’ve confirmed the timeout issue is on our end, and we’re working on a fix. We’ll update you as soon as it’s resolved.

---

<div class="post-metadata">

**Author:** ![Qirall79](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/qirall79/32/14992_2.png) [@Qirall79](https://forum.codecrafters.io/u/Qirall79)\
**Post date:** [July 27, 2025, 6:00pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/8 "2025-07-27T18:00:09Z")

</div>

Thank you !

---

<div class="post-metadata">

**Author:** ![andy1li](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/andy1li/32/10_2.png) [@andy1li](https://forum.codecrafters.io/u/andy1li)\
**Post date:** [August 26, 2025, 10:44pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/9 "2025-08-26T22:44:26Z")

</div>

@Qirall79 The timeout issue should now be resolved. Please let us know if you’re still running into it.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/7/0/700657133935c15703e22c7c8870394f6e6dc27d.svg) [@system](https://forum.codecrafters.io/u/system)\
**Post date:** [September 2, 2025, 10:45pm UTC](https://forum.codecrafters.io/t/sqlite-retrieving-data-using-index-times-out-when-testing/14024/11 "2025-09-02T22:45:28Z")

</div>

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.