Stucks on Multiple persistent connections (UL1)

Challenges
1

I’m stuck on Stage #UL1:.

I am working with http-server challenge using Typescript and get stuck on stage UL1. I think I got the problem of when the test sends two consecutive requests quickly, the server writes both responses to the same socket in quick succession. Since the socket doesn’t automatically split these responses, the client ends up receiving both responses combined into one

I’ve tried setNoDelay to true, use setImmediate and setTimeout.

Here are my logs:

[tester::#UL1] Running tests for Stage #UL1 (Persistent Connections - Multiple persistent connections)
[tester::#UL1] Running program
[tester::#UL1] $ ./your_program.sh
[tester::#UL1] Creating 2 persistent connections
[your_program] Logs from your program will appear here!
[tester::#UL1] Sending first set of requests
[tester::#UL1] $ curl --http1.1 -v http://localhost:4221/user-agent -H "User-Agent: blueberry/apple" --next http://localhost:4221/echo/blueberry
[tester::#UL1] client-2: > GET /echo/blueberry HTTP/1.1
[tester::#UL1] client-2: > Host: localhost:4221
[tester::#UL1] client-2: >
[tester::#UL1] client-2: Sent bytes: "GET /echo/blueberry HTTP/1.1\r\nHost: localhost:4221\r\n\r\n"
[your_program] Received data: GET /echo/blueberry HTTP/1.1
[your_program] Host: localhost:4221
[your_program]
[your_program]
[your_program] urlParams [ "echo", "blueberry" ]
[your_program] message blueberry
[your_program] acceptedEncoding:  [ "" ]
[your_program] response:
[your_program]  HTTP/1.1 200 OK
[your_program] Content-Type: text/plain
[your_program] Content-Length: 9
[your_program]
[your_program] blueberry
[your_program]
[tester::#UL1] client-2: Received bytes: "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 9\r\n\r\nblueberry"
[tester::#UL1] < HTTP/1.1 200 OK
[tester::#UL1] < Content-Type: text/plain
[tester::#UL1] < Content-Length: 9
[tester::#UL1] <
[tester::#UL1] < blueberry
[tester::#UL1] <
[tester::#UL1] * Connection #1 to host localhost left intact
[tester::#UL1] Received response with 200 status code
[tester::#UL1] ✓ Content-Type header is present
[tester::#UL1] ✓ Content-Length header is present
[tester::#UL1] ✓ Body is correct
[tester::#UL1] client-1: > GET /user-agent HTTP/1.1
[tester::#UL1] client-1: > Host: localhost:4221
[tester::#UL1] client-1: > User-Agent: blueberry/apple
[tester::#UL1] client-1: >
[tester::#UL1] client-1: Sent bytes: "GET /user-agent HTTP/1.1\r\nHost: localhost:4221\r\nUser-Agent: blueberry/apple\r\n\r\n"
[your_program] Received data: GET /user-agent HTTP/1.1
[your_program] Host: localhost:4221
[your_program] User-Agent: blueberry/apple
[your_program]
[your_program]
[your_program] urlParams [ "user-agent" ]
[your_program] response:
[your_program]  HTTP/1.1 200 OK
[your_program] Content-Type: text/plain
[your_program] Content-Length: 15
[your_program]
[your_program] blueberry/apple
[your_program]
[tester::#UL1] client-1: Received bytes: "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 15\r\n\r\nblueberry/apple"
[tester::#UL1] < HTTP/1.1 200 OK
[tester::#UL1] < Content-Type: text/plain
[tester::#UL1] < Content-Length: 15
[tester::#UL1] <
[tester::#UL1] < blueberry/apple
[tester::#UL1] <
[tester::#UL1] * Connection #0 to host localhost left intact
[tester::#UL1] Received response with 200 status code
[tester::#UL1] ✓ Content-Type header is present
[tester::#UL1] ✓ Content-Length header is present
[tester::#UL1] ✓ Body is correct
[tester::#UL1] Sending second set of requests
[tester::#UL1] $ curl --http1.1 -v http://localhost:4221/user-agent -H "User-Agent: blueberry/apple" --next http://localhost:4221/echo/blueberry
[tester::#UL1] * Re-using existing connection with host localhost
[tester::#UL1] client-1: > GET /user-agent HTTP/1.1
[tester::#UL1] client-1: > Host: localhost:4221
[tester::#UL1] client-1: > User-Agent: blueberry/apple
[tester::#UL1] client-1: >
[tester::#UL1] client-1: Sent bytes: "GET /user-agent HTTP/1.1\r\nHost: localhost:4221\r\nUser-Agent: blueberry/apple\r\n\r\n"
[your_program] Received data: GET /user-agent HTTP/1.1
[your_program] Host: localhost:4221
[your_program] User-Agent: blueberry/apple
[your_program]
[your_program]
[your_program] urlParams [ "user-agent" ]
[your_program] response:
[your_program]  HTTP/1.1 200 OK
[your_program] Content-Type: text/plain
[your_program] Content-Length: 15
[your_program]
[your_program] blueberry/apple
[your_program]
[tester::#UL1] Failed to read response:
[tester::#UL1] Received: "\r\nHTTP/1.1 200 OK\r\nContent-Typ"
[tester::#UL1]            ^ error
[tester::#UL1] Error: Expected 'HTTP/1.1', Received: "\r\nHTTP/1"
[tester::#UL1] Test failed
[tester::#UL1] Terminating program
[tester::#UL1] Program terminated successfully

And here’s a snippet of my code:

import * as net from "net";
import fs from "node:fs";
import createGzip from "node:zlib";

// You can use print statements as follows for debugging, they'll be visible when running tests.
console.log("Logs from your program will appear here!");

const CRLF = "\r\n";

const USER_AGENT_PATH = "/user-agent";
const ECHO_PATH = "/echo";
const FILE_PATH = "/files";
const DEFAULT_PATH = "/";

// Uncomment this to pass the first stage
const server = net.createServer(
  { keepAlive: true, noDelay: true },
  (socket) => {
    // socket.write(Buffer.from(`HTTP/1.1 200 OK\r\n\r\n`));

    const requestQueue = [];
    socket.setKeepAlive(true);

    const buildResponse = (
      httpVersion: string,
      statusCode: number,
      headers?: { [key: string]: string },
      body?: string,
    ) => {
      const statusCodeResMapping: Record<number, string> = {
        200: "OK",
        201: "Created",
        404: "Not Found",
        500: "Internal Server Error",
      };

      const statusCodeRes = statusCodeResMapping[statusCode];
      let response = `${httpVersion} ${statusCode} ${statusCodeRes}${CRLF}`;

      if (headers) {
        for (const [key, value] of Object.entries(headers)) {
          response += `${key}: ${value}${CRLF}`;
        }
      }

      if (body) {
        response += `Content-Length: ${Buffer.byteLength(body)}${CRLF}`;
        response += `${CRLF}${body}`;
      }

      response += CRLF;

      console.log("response: \n", response);

      return response;
    };

    const extractUrlParams = (urlPathStr: string) => {
      return urlPathStr.split("/").slice(1);
    };

    const parseHeaders = (headers: string[]) => {
      let results: Record<string, string> = {};

      headers.forEach((header) => {
        const [key, value] = header.split(": ");
        results[key] = value;
      });

      return results;
    };

    const writeData = async (
      socket: net.Socket,
      data: string | Buffer | Uint8Array,
      delay = 1000,
    ) => {
      setTimeout(() => socket.write(data), delay);
    };

    // Handle data event
    socket.on("data", async (data) => {
      console.log("Received data:", data.toString());
      const receivedDataStr = data.toString();
      const requestLine = receivedDataStr.split(CRLF)[0];
      const httpMethod = requestLine.split(" ")[0];
      const requestPath = requestLine.split(" ")[1];
      const httpVersion = requestLine.split(" ")[2];
      const headers = receivedDataStr.split(CRLF).slice(1, -2);
      const requestBody = receivedDataStr.split(CRLF).slice(-1)[0];

      const urlParams = extractUrlParams(requestPath);
      const urlOrigin = `/${urlParams[0]}`;
      const headersObj = parseHeaders(headers);
      console.log("urlParams", urlParams);

      switch (urlOrigin) {
        case DEFAULT_PATH:
          await writeData(socket, Buffer.from(`HTTP/1.1 200 OK\r\n\r\n`));
          break;
        case ECHO_PATH:
          const message = urlParams[1];
          console.log("message", message);
          const acceptedEncodingStr = headersObj["Accept-Encoding"] ?? "";
          const acceptedEncodings = acceptedEncodingStr
            .split(",")
            .map((e) => e.trim());
          console.log("acceptedEncoding: ", acceptedEncodings);
          const allowedEncodings = "gzip";
          // Filter allowed encoding from accepted encodings
          if (acceptedEncodings.includes(allowedEncodings)) {
            const compressedMessage = Bun.gzipSync(
              Buffer.from(message, "utf8"),
            );
            await writeData(
              socket,
              buildResponse(httpVersion, 200, {
                "Content-Type": "text/plain",
                "Content-Encoding": "gzip",
                "content-Length": compressedMessage.length.toString(),
              }),
            );
            await writeData(socket, compressedMessage);
          } else {
            await writeData(
              socket,
              Buffer.from(
                buildResponse(
                  httpVersion,
                  200,
                  {
                    "Content-Type": "text/plain",
                  },
                  message,
                ),
              ),
            );
          }
          break;
        case USER_AGENT_PATH:
          const userAgent = headersObj["User-Agent"];
          await writeData(
            socket,
            Buffer.from(
              buildResponse(
                httpVersion,
                200,
                { "Content-Type": "text/plain" },
                userAgent,
              ),
            ),
          );
          break;
        case FILE_PATH:
          const fileName = `/${urlParams.slice(1).join("/")}`;
          console.log("fileName", fileName);
          const args = process.argv.slice(2);
          const [___, absPath] = args;
          console.log("absPath", absPath);
          const filePath = absPath + "/" + fileName;
          if (httpMethod === "GET") {
            try {
              const content = fs.readFileSync(filePath);
              console.log("content", content);
              await writeData(
                socket,
                buildResponse(
                  httpVersion,
                  200,
                  { "Content-Type": "application/octet-stream" },
                  content.toString(),
                ),
              );
            } catch (_) {
              await writeData(socket, buildResponse(httpVersion, 404));
            }
          } else if (httpMethod === "POST") {
            try {
              fs.writeFileSync(filePath, requestBody);
              await writeData(socket, buildResponse(httpVersion, 201));
            } catch (error) {
              console.error(error);
            }
          }
          break;
        default:
          await writeData(
            socket,
            Buffer.from(`HTTP/1.1 404 Not Found\r\n\r\n`),
          );
          break;
      }
    });

    socket.on("drain", () => {
      console.log("socket on drain");
    });

    socket.on("close", () => {
      console.log("socket on close");
      socket.end();
    });
  },
);

server.listen(4221, "localhost");
4

Hey @chuanhd, could you share a bit more context around the setTimeout in writeData? Curious to understand what it’s used for.

As for the error:

image
image1328×226 63.1 KB

It looks like it’s caused by this line:

image
image1390×696 66.9 KB

Specifically, the extra CRLF after the body is incorrect and got picked up by the next request/response cycle.

6

Thanks for your response.

By using setTimeout() with a specified delay, my idea is to separate the writes to the socket, ensuring each write happens independently in its own cycle of the event loop. This way, the client will receive separate responses rather than combined ones, even when requests come in rapidly.

Thank you! You’re absolutely right. I made a mistake while refactoring the buildResponse function. The code in the red box should actually be placed in the else clause.

       if (body) {
        response += `Content-Length: ${Buffer.byteLength(body)}${CRLF}`;
        response += `${CRLF}${body}`;
      } else {
        response += CRLF;
      }
1 Like