Http server stuck at get a file stage

[khchehab]

I’m stuck on Stage #AP6.

The weird thing is when I was at the stage before it (#EJ5), the test succeeded normally, but after adding the part about the files endpoint, the files endpoint test succeeds and the previous stage no longer succeeds.

I’ve tried getting the last code that succeeded and just add the if condition for the files endpoint but it didn’t work.

Has anyone faced this in Go (or any other language)?

Here are my logs:

remote: [tester::#AP6] Running tests for Stage #AP6 (Get a file)
remote: [tester::#AP6] Running program
remote: [tester::#AP6] $ ./your_server.sh --directory /tmp/data/codecrafters.io/http-server-tester/
remote: [tester::#AP6] Testing existing file
remote: [tester::#AP6] Creating file apple_grape_blueberry_mango in /tmp/data/codecrafters.io/http-server-tester/
remote: [tester::#AP6] File Content: "pineapple pineapple pineapple strawberry strawberry blueberry banana banana"
remote: [your_program] Logs from your program will appear here!
remote: [tester::#AP6] Connected to localhost port 4221
remote: [tester::#AP6] $ curl -v http://localhost:4221/files/apple_grape_blueberry_mango
remote: [tester::#AP6] > GET /files/apple_grape_blueberry_mango HTTP/1.1
remote: [tester::#AP6] > Host: localhost:4221
remote: [tester::#AP6] > 
remote: [tester::#AP6] Sent bytes: "GET /files/apple_grape_blueberry_mango HTTP/1.1\r\nHost: localhost:4221\r\n\r\n"
remote: [tester::#AP6] Received bytes: "HTTP/1.1 200 OK\r\nContent-Type: application/octet-stream\r\nContent-Length: 75\r\n\r\npineapple pineapple pineapple strawberry strawberry blueberry banana banana"
remote: [tester::#AP6] < HTTP/1.1 200 OK
remote: [tester::#AP6] < Content-Type: application/octet-stream
remote: [tester::#AP6] < Content-Length: 75
remote: [tester::#AP6] < 
remote: [tester::#AP6] < pineapple pineapple pineapple strawberry strawberry blueberry banana banana
remote: [tester::#AP6] < 
remote: [tester::#AP6] Received response with 200 status code
remote: [tester::#AP6] ✓ Content-Type header is present
remote: [tester::#AP6] ✓ Content-Length header is present
remote: [tester::#AP6] ✓ Body is correct
remote: [your_program] filename: apple_grape_blueberry_mango
remote: [tester::#AP6] First test passed.
remote: [tester::#AP6] Testing non existent file returns 404
remote: [tester::#AP6] Connected to localhost port 4221
remote: [tester::#AP6] $ curl -v http://localhost:4221/files/non-existentstrawberry_orange_mango_orange
remote: [tester::#AP6] > GET /files/non-existentstrawberry_orange_mango_orange HTTP/1.1
remote: [tester::#AP6] > Host: localhost:4221
remote: [tester::#AP6] > 
remote: [tester::#AP6] Sent bytes: "GET /files/non-existentstrawberry_orange_mango_orange HTTP/1.1\r\nHost: localhost:4221\r\n\r\n"
remote: [your_program] filename: non-existentstrawberry_orange_mango_orange
remote: [tester::#AP6] Received bytes: "HTTP/1.1 404 Not Found\r\n\r\n"
remote: [tester::#AP6] < HTTP/1.1 404 Not Found
remote: [tester::#AP6] < 
remote: [tester::#AP6] Received response with 404 status code
remote: [tester::#AP6] Test passed.
remote: [tester::#AP6] Terminating program
remote: [tester::#AP6] Program terminated successfully
remote: 
remote: [tester::#EJ5] Running tests for Stage #EJ5 (Concurrent connections)
remote: [tester::#EJ5] Running program
remote: [tester::#EJ5] $ ./your_server.sh
remote: [tester::#EJ5] Creating 4 parallel connections
remote: [tester::#EJ5] Creating connection 1
remote: [your_program] Logs from your program will appear here!
remote: [tester::#EJ5] Creating connection 2
remote: [tester::#EJ5] Creating connection 3
remote: [tester::#EJ5] Creating connection 4
remote: [tester::#EJ5] client-1: $ curl -v http://localhost:4221/
remote: [tester::#EJ5] client-1: > GET / HTTP/1.1
remote: [tester::#EJ5] client-1: > Host: localhost:4221
remote: [tester::#EJ5] client-1: > 
remote: [tester::#EJ5] client-1: Sent bytes: "GET / HTTP/1.1\r\nHost: localhost:4221\r\n\r\n"
remote: [tester::#EJ5] Failed to read response: 
remote: [tester::#EJ5] Received: "" (no content received)
remote: [tester::#EJ5]            ^ error
remote: [tester::#EJ5] Error: Expected: HTTP-version, Received: ""
remote: [tester::#EJ5] Test failed
remote: [tester::#EJ5] Terminating program
remote: [tester::#EJ5] Program terminated successfully

And here’s a snippet of my code:

func doConnection(conn net.Conn, directory string) {
	defer func(conn net.Conn) {
		if err := conn.Close(); err != nil {
			fmt.Printf("error closing the connection: %v", err.Error())
		}
	}(conn)

	var err error
	var n int
	var bytesBuffer bytes.Buffer
	temp := make([]byte, 10)

	for {
		if n, err = conn.Read(temp); err != nil && err != io.EOF {
			return
		}

		if n == 0 {
			break
		}

		bytesBuffer.Write(temp[:n])
		if n < len(temp) {
			break
		}
	}

	if bytesBuffer.Len() == 0 {
		return
	}

	buffer := bytesBuffer.Bytes()

	var requestSepIndex int
	if requestSepIndex = bytes.Index(buffer, []byte{'\r', '\n'}); requestSepIndex == -1 {
		return
	}

	requestComponents := bytes.Split(buffer[0:requestSepIndex:requestSepIndex], []byte{' '})
	//method := string(requestComponents[0])
	path := string(requestComponents[1])
	//version := string(requestComponents[2])

	var headerSepIndex int
	if headerSepIndex = bytes.Index(buffer, []byte{'\r', '\n', '\r', '\n'}); headerSepIndex == -1 {
		return
	}

	headers := make(map[string]string)
	if headerSepIndex+4 < len(buffer) {
		headerEntries := bytes.Split(buffer[requestSepIndex+2:headerSepIndex:headerSepIndex], []byte{'\r', '\n'})
		var splitHeader [][]byte
		for _, headerEntry := range headerEntries {
			if splitHeader = bytes.Split(headerEntry, []byte{':', ' '}); len(splitHeader) == 2 {
				headers[string(splitHeader[0])] = string(splitHeader[1])
			}
		}
	}

	var output string
	if path == "/" {
		output = "HTTP/1.1 200 OK\r\n\r\n"
	} else if i := strings.Index(path, "/echo/"); i != -1 {
		param := path[i+6:]
		output = fmt.Sprintf("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %d\r\n\r\n%s", len(param), param)
	} else if path == "/user-agent" {
		userAgent := headers["User-Agent"]
		output = fmt.Sprintf("HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: %d\r\n\r\n%s", len(userAgent), userAgent)
	} else if i := strings.Index(path, "/files/"); i != -1 {
		filename := path[i+7:]
		fmt.Println("filename:", filename)
		dirFs := os.DirFS(directory)

		var f fs.File
		if f, err = dirFs.Open(filename); err != nil {
			if errors.Is(err, fs.ErrNotExist) {
				output = "HTTP/1.1 404 Not Found\r\n\r\n"
			} else {
				output = "HTTP/1.1 400 Bad Request\r\n\r\n"
			}
		} else {
			if fc, err := io.ReadAll(f); err != nil {
				output = "HTTP/1.1 400 Bad Request\r\n\r\n"
			} else {
				output = fmt.Sprintf("HTTP/1.1 200 OK\r\nContent-Type: application/octet-stream\r\nContent-Length: %d\r\n\r\n%s", len(fc), string(fc))
			}
		}
	} else {
		output = "HTTP/1.1 404 Not Found\r\n\r\n"
	}

	if _, err = conn.Write([]byte(output)); err != nil {
		return
	}
}

[khchehab]

For some reason, when reading the request directly into one big buffer, all of the tests as passing.

I replaced

    var err error
	var n int
	var bytesBuffer bytes.Buffer
	temp := make([]byte, 10)

	for {
		if n, err = conn.Read(temp); err != nil && err != io.EOF {
			return
		}

		if n == 0 {
			break
		}

		bytesBuffer.Write(temp[:n])
		if n < len(temp) {
			break
		}
	}

	if bytesBuffer.Len() == 0 {
		return
	}

	buffer := bytesBuffer.Bytes()

with

	buffer := make([]byte, 1024)
	_, err := conn.Read(buffer)
	if err != nil {
		return
	}

I wanted to try and read the request in chunks to take into consideration reading a request bigger than the buffer size, but for some reason it’s not working.

[Thomas-Dvorak]

I also have the same problem and error. I passed the last stage, completed this stage, but fail with the previous stage for some reason.
I have been looking into my code at least and have not been able to find the solution so far. I am using C# and here’s my code so far:

using System.Net;
using System.Net.Sockets;
using System.Text;

TcpListener server = new(IPAddress.Any, 4221);
Console.WriteLine("Connection to server accepted.");
Console.WriteLine("Starting server...");
server.Start();
Console.WriteLine("Server started.");
Console.WriteLine("Waiting for a connection...");

// support multiple sockets
while (true) {
    TcpClient client = server.AcceptTcpClient();
    Console.WriteLine("Connection accepted. New socket created.");
    _ = Task.Run(async () => await HandleClient(client));
    // run sockets together
}

Task HandleClient(TcpClient client) {
    // handle each client connected to server
    byte[] response = Encoding.UTF8.GetBytes(GetResponseFromMessage(client));
    client.GetStream().Write(response, 0, response.Length);
    client.Close();
    return Task.CompletedTask;
}
// end of multiple sockets

string GetResponseFromMessage(TcpClient client) {
    // reads the lines
    // gets path and version
    // prepares response based on path and header info
    var lines = RecieveMessageFromClient(client).Split("\r\n");
    var headerParts = lines[0].Split(" ");
    var path = headerParts[1];
    var httpVersion = headerParts[2];
    string[] args = Environment.GetCommandLineArgs();
    // get command line arguments (i.e. --directory <directory>)
    // args = ["./your_server.sh", "--directory", "<directory>"]
    if (args[1].Equals("--directory")) {
        if (path.StartsWith("/files/")) {
            // read the file if it exists and we're in the /file/ directory
            // error handling for nonexistent files or directories
            try {
                string fileName = path[7..];
                StreamReader reader = new(Path.Combine(args[2], fileName));
                string data = reader.ReadToEnd();
                return PrepareFileReadResponse(httpVersion, data);
            } catch (Exception e) {
                Console.WriteLine("Error in reading file. File or directory does not exist.");
                return PrepareNotFoundResponse(httpVersion);
            }
        } else {
            // you need to be in /files/
            return PrepareNotFoundResponse(httpVersion);
        }
    } else if (path.StartsWith("/echo/")) {
        // print {text} in from ~/echo/{text}
        string text = path[6..];
        return PrepareTextResponse(httpVersion, text);
    } else if (path.StartsWith("/user-agent")) {
        foreach (var line in lines) {
            if (line.StartsWith("User-Agent:")) {
                string text = line[12..];
                text = text[..^4];
                return PrepareTextResponse(httpVersion, text);
            } else {
                return PrepareNotFoundResponse(httpVersion);
            }
        }
    } else if (path.Equals("/")) {
        // we could be in the root directory
        return PrepareOKConnectionResponse(httpVersion, false);
    } 
    return PrepareNotFoundResponse(httpVersion);
}

// next 4 functions prepare responses to be sent back to the user

string PrepareTextResponse(string httpVersion, string text) {
    return $"{PrepareOKConnectionResponse(httpVersion, true)}Content-Type: text/plain\r\nContent-Length: {text.Length}\r\n\r\n{text}";
}

string PrepareNotFoundResponse(string httpVersion) {
    return $"{httpVersion} 404 Not Found\r\n\r\n";
}

string PrepareOKConnectionResponse(string httpsVersion, bool includeHeaders) {
    return $"{httpsVersion} 200 OK\r\n" + (includeHeaders ? "" : "\r\n");
}

string PrepareFileReadResponse(string httpsVersion, string fileContents) {
    return $"{PrepareOKConnectionResponse(httpsVersion, true)}Content-Type: application/octet-stream\r\nContent-Length: {fileContents.Length}\r\n\r\n{fileContents}";
}

string RecieveMessageFromClient(TcpClient client) {
    // recieve command/message from client
    // make a buffer for recieving the data
    var buffer = new byte[1024];
    int bytesRead = client.GetStream().Read(buffer, 0, buffer.Length);
    // turn it into a string
    return Encoding.UTF8.GetString(buffer, 0, bytesRead);
}

I originally thought it would be in my GetResponseFromMessage(client), but all paths return a value. I tried working with HandleClient(client), but I got nowhere.
My buffer request size is byte[1024] also.
I am continuing to look into how to fix this problem and will try to help you if I find the solution.
UPDATE: I thought it may be an async problem, but it’s not.
UPDATE II: The error arises in my code at:
byte[] response = Encoding.UTF8.GetBytes(GetResponseFromMessage(client));

I got it working and all the tests passed. What I had wrong was that I was checking if args[1] was equal to --directory, but did not check if args.Length > 1. It now works.

[rohitpaulk]

Thanks for sharing the solution that worked for you!