HTTP Server #CR8: Gzip compression

inner · May 11, 2024, 8:07am

I’m stuck on gzip compression stage 3.

I’ve also tried sending the gzip compressed body as hex, base64. I’ve provided my current version of the code snippet below.

Here are my logs:

inner@afterlife:/mnt/d/projects/codecrafters-http-server-csharp$ codecrafters test
Initiating test run...

⚡ This is a turbo test run. https://codecrafters.io/turbo

Running tests. Logs should appear shortly...

remote: [compile] MSBuild version 17.3.2+561848881 for .NET        
remote: [compile]   Determining projects to restore...        
remote: [compile]   Restored /app/codecrafters-http-server.csproj (in 90 ms).        
remote: [compile]   codecrafters-http-server -> /app/bin/Release/net6.0/codecrafters-http-server.dll        
remote: [compile]
remote: [compile] Build succeeded.        
remote: [compile]     0 Warning(s)        
remote: [compile]     0 Error(s)        
remote: [compile]
remote: [compile] Time Elapsed 00:00:03.17        
remote: [compile]Compilation successful.
remote: 
remote: Debug = true        
remote: 
remote: [http-compression-3] Running tests for HTTP Compression > Stage #3: Gzip compression
remote: [http-compression-3] Running program
remote: [http-compression-3] $ ./your_server.sh
remote: [your_program] Starting custom HTTP Server.        
remote: [your_program] Custom HTTP Server started.        
remote: [http-compression-3] Connected to localhost port 4221
remote: [http-compression-3] $ curl -v -X GET http://localhost:4221/echo/blueberry -H "Accept-Encoding: gzip"
remote: [http-compression-3] > GET /echo/blueberry HTTP/1.1
remote: [http-compression-3] > Host: localhost:4221
remote: [http-compression-3] > Accept-Encoding: gzip
remote: [http-compression-3] > 
remote: [http-compression-3] Sent bytes: "GET /echo/blueberry HTTP/1.1\r\nHost: localhost:4221\r\nAccept-Encoding: gzip\r\n\r\n"
remote: [your_program] TCP Connection 127.0.0.1:38108 established!        
remote: [your_program] Hex string: 1F8B08000000000000034BCA294D4D4A2D2AAA0400D3241E7909000000        
remote: [http-compression-3] Received bytes: "HTTP/1.1 200 OK\r\nContent-Encoding: gzip\r\nContent-Type: text/plain\r\nContent-Length: 29\r\n\r\n\x1f?\b\x00\x00\x00\x00\x00\x00\x03K?)MMJ-*?\x04\x00?$\x1ey\t\x00\x00\x00"
remote: [http-compression-3] < HTTP/1.1 200 OK
remote: [http-compression-3] < Content-Encoding: gzip
remote: [http-compression-3] < Content-Type: text/plain
remote: [http-compression-3] < Content-Length: 29
remote: [http-compression-3] < 
remote: [http-compression-3] < <Binary Content>
remote: [http-compression-3] < 
remote: [http-compression-3] Received response with 200 status code
remote: [http-compression-3] ✓ Content-Encoding header is present
remote: [http-compression-3] ✓ Content-Length header is present
remote: [http-compression-3] Failed to decode gzip: Failed to decompress data: gzip: invalid header
remote: [http-compression-3] Test failed
remote: [http-compression-3] Terminating program
remote: [http-compression-3] Program terminated successfully
remote: 
remote: View our article on debugging test failures: https://codecrafters.io/debug

And here’s a snippet of my code:

            var echoMessage = serverRequest.Path.Replace(EchoEndpointPath, string.Empty);

            var acceptEncoding = serverRequest.Headers.Where(x =>
                    string.Equals(x.Key, "Accept-Encoding", StringComparison.InvariantCultureIgnoreCase))
                .Select(x => x.Value)
                .FirstOrDefault()?
                .Split(',')
                .Select(x => x.Trim());

            var containsGzipEncoding = acceptEncoding?
                .Contains("gzip", StringComparer.InvariantCultureIgnoreCase);

            var includeGzip = containsGzipEncoding.HasValue && containsGzipEncoding.Value;

            sb.Append("HTTP/1.1 200 OK\r\n");

            if (includeGzip)
            {
                byte[] compressedData;
                using (var memoryStream = new MemoryStream())
                {
                    using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress))
                    {
                        var stringBytes = Encoding.UTF8.GetBytes(echoMessage);
                        gzipStream.Write(stringBytes, 0, stringBytes.Length);
                        gzipStream.Flush();
                    }

                    compressedData = memoryStream.ToArray();

                    Console.WriteLine($"Hex string: {BitConverter.ToString(compressedData).Replace("-", string.Empty)}");
                }
                
                sb.Append("Content-Type: text/plain\r\n");
                sb.Append("Content-Encoding: gzip\r\n");
                sb.Append($"Content-Length: {compressedData.Length}\r\n\r\n");
                sb.Append(Encoding.UTF8.GetString(compressedData));
            }
            else
            {
                sb.Append("Content-Type: text/plain\r\n");
                sb.Append($"Content-Length: {echoMessage.Length}\r\n\r\n");
                sb.Append(echoMessage);
            }

ValentinJub · May 11, 2024, 3:22pm

I’m getting the same error in the test, I’ve also tried hex, base64 or binary but they all result in the same error.

[http-compression-3] $ curl -v -X GET http://localhost:4221/echo/banana -H "Accept-Encoding: gzip"
[http-compression-3] > GET /echo/banana HTTP/1.1
[http-compression-3] > Host: localhost:4221
[http-compression-3] > Accept-Encoding: gzip
[http-compression-3] >
[http-compression-3] Sent bytes: "GET /echo/banana HTTP/1.1\r\nHost: localhost:4221\r\nAccept-Encoding: gzip\r\n\r\n"
[your_program] Parsed & Split Data;  [
[your_program]   'GET /echo/banana HTTP/1.1',
[your_program]   'Host: localhost:4221',
[your_program]   'Accept-Encoding: gzip',
[your_program]   '',
[your_program]   ''
[your_program] ]
[your_program] 11/05/24 15:19:06 Extracted path banana
[your_program] Gzipped data: <Buffer 1f 8b 08 00 00 00 00 00 00 03 4b 4a cc 03 42 00 cf 67 8b 03 06 00 00 00>
[your_program] Gzipped data as a Base64 string: H4sIAAAAAAAAA0tKzANCAM9niwMGAAAA
[your_program] 11/05/24 15:19:06 HTTP/1.1 200 OK
[your_program] Content-Encoding: gzip
[your_program] Content-Type: text/plain
[your_program] Content-Length: 50
[your_program]
[your_program] 1f8b08000000000000034b4acc034200cf678b0306000000
[your_program]
[http-compression-3] Received bytes: "HTTP/1.1 200 OK\r\nContent-Encoding: gzip\r\nContent-Type: text/plain\r\nContent-Length: 50\r\n\r\n1f8b08000000000000034b4acc034200cf678b0306000000\r\n"
[http-compression-3] < HTTP/1.1 200 OK
[http-compression-3] < Content-Encoding: gzip
[http-compression-3] < Content-Type: text/plain
[http-compression-3] < Content-Length: 50
[http-compression-3] <
[http-compression-3] < <Binary Content>
[http-compression-3] <
[http-compression-3] Received response with 200 status code
[http-compression-3] ✓ Content-Encoding header is present
[http-compression-3] ✓ Content-Length header is present
[http-compression-3] Failed to decode gzip: Failed to decompress data: gzip: invalid header
[http-compression-3] Test failed
[http-compression-3] Terminating program
[http-compression-3] Program terminated successfully

function gzipString(inputString) {
    return new Promise((resolve, reject) => {
        // Convert the string to a Buffer since zlib works with binary data
        const buffer = Buffer.from(inputString, 'utf-8');
        zlib.gzip(buffer, (err, result) => {
            if (err) {
                reject(err);
            } else {
                resolve(result);
            }
        });
    });
}

    async _send200(data, content_type, encoding) {
        data = data ? data : '';
        if (encoding[0] === 'gzip') {
            try {
                const gzippedData = await gzipString(data)
                encoding = `Content-Encoding: ${encoding[0]}${CRLF}`
                const encoded = gzippedData.toString('hex') + CRLF
                console.log('Gzipped data:', gzippedData);
                console.log('Gzipped data as a Base64 string:', gzippedData.toString('base64'));
                return `${HTTPResponse[200]}${encoding}Content-Type: ${content_type}${CRLF}Content-Length: ${encoded.length}${CRLF}${CRLF}${encoded}`;
            } catch (e) {
                console.error(e);
                return `${HTTPResponse[500]}Content-Type: text/plain${CRLF}Content-Length: 21${CRLF}${CRLF}Internal Server Error${CRLF}`;
            }
        } else {
            return `${HTTPResponse[200]}Content-Type: ${content_type}${CRLF}Content-Length: ${data.length}${CRLF}${CRLF}${data}${CRLF}`;
        }
    }

ValentinJub · May 11, 2024, 3:46pm

Okay I managed to pass the test, my issue was that the gzip encoded data that I was passing wasn’t encoded properly in my response. The test is expecting gzip encoded data that it will open with gunzip and check that it returns the same string in the echo

inner · May 11, 2024, 5:01pm

thanks for the hint on gunzip. Tests passed

noe-lc · May 14, 2024, 9:18pm

Hey! could you please post a snippet of your solution? I’ve already checked that my encoded response indeed returns the same input string after decompression - yet I am still getting the same error…

rohitpaulk · May 14, 2024, 9:26pm

@noe-lc you’ll find code snippets from users who passed the stage in the “Code Examples” tab:

noe-lc · May 14, 2024, 10:11pm

Thanks, I did check the examples - the logic looks is exactly same but just some implementation differences. Still do not know what in my code is wrong ,so I wanted to see how someone with the exact same issue worked around it

inner · May 15, 2024, 2:57am

In my case, the Encoding.UTF8.GetString(compressedData) was impacting the result as this function call was converting the compatible gzip output to incompatible

noe-lc · May 15, 2024, 3:30pm

For those stumbling upon this error while using TS and plain socket writes, you need to write the response in two parts:

First write the startLine + headers: socket.write(startLineAndHeaders)
Write the compressed data socket.write(compressedData)
End the socket with socket.end()

This is done so that the client is prepared to receive gzip encoded data.

stevenh-gh · May 20, 2024, 4:15am

Thanks! This worked for me.
I was under the impression that I needed to send the hex representation of gzip zlib.gzipSync(string).toString('hex') and I would write the entire response with interpolated values. It seems I didn’t need the toString('hex') after all I just needed a 2nd write.

system · May 25, 2024, 4:16am

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.

system · June 3, 2024, 7:51pm

Note: I’ve updated the title of this post to include the stage ID (#CR8). You can learn about the stages rename here: Upcoming change: Stages overhaul.

Topic		Replies	Views
HTTP Compression stage #DF4 Challenges challenge:http-server	3	172	June 4, 2024
Failed to decode gzip: Failed to decompress data: gzip: invalid header - Rust Challenges challenge:http-server	2	169	June 26, 2024
Failed to decode gzip: Failed to decompress data: gzip: invalid header JAVASCRIPT Challenges challenge:http-server	2	172	July 3, 2024
Stuck on stage #CR8 Challenges challenge:http-server	2	62	July 12, 2024
HTTP Server (#DF4) JavaScript Challenges challenge:http-server	4	189	June 4, 2024

HTTP Server #CR8: Gzip compression

Related topics