# HTTP Server #CR8: Gzip compression

**URL:** <https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348>\
**Category:** Challenges\
**Tags:** challenge:http-server\
**Created:** [May 11, 2024, 8:07am UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348 "2024-05-11T08:07:50Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![inner](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/inner/32/4695_2.png) [@inner](https://forum.codecrafters.io/u/inner)\
**Post date:** [May 11, 2024, 8:07am UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/1 "2024-05-11T08:07:50Z")

</div>

I’m stuck on gzip compression stage 3.

I’ve also tried sending the gzip compressed body as hex, base64. I’ve provided my current version of the code snippet below.

Here are my logs:

```auto
inner@afterlife:/mnt/d/projects/codecrafters-http-server-csharp$ codecrafters test
Initiating test run...

⚡ This is a turbo test run. https://codecrafters.io/turbo

Running tests. Logs should appear shortly...

remote: [compile] MSBuild version 17.3.2+561848881 for .NET        
remote: [compile] Determining projects to restore...        
remote: [compile] Restored /app/codecrafters-http-server.csproj (in 90 ms).        
remote: [compile] codecrafters-http-server -> /app/bin/Release/net6.0/codecrafters-http-server.dll        
remote: [compile]
remote: [compile] Build succeeded.        
remote: [compile] 0 Warning(s)        
remote: [compile] 0 Error(s)        
remote: [compile]
remote: [compile] Time Elapsed 00:00:03.17        
remote: [compile]Compilation successful.
remote: 
remote: Debug = true        
remote: 
remote: [http-compression-3] Running tests for HTTP Compression > Stage #3: Gzip compression
remote: [http-compression-3] Running program
remote: [http-compression-3] $ ./your_server.sh
remote: [your_program] Starting custom HTTP Server.        
remote: [your_program] Custom HTTP Server started.        
remote: [http-compression-3] Connected to localhost port 4221
remote: [http-compression-3] $ curl -v -X GET http://localhost:4221/echo/blueberry -H "Accept-Encoding: gzip"
remote: [http-compression-3] > GET /echo/blueberry HTTP/1.1
remote: [http-compression-3] > Host: localhost:4221
remote: [http-compression-3] > Accept-Encoding: gzip
remote: [http-compression-3] > 
remote: [http-compression-3] Sent bytes: "GET /echo/blueberry HTTP/1.1\r\nHost: localhost:4221\r\nAccept-Encoding: gzip\r\n\r\n"
remote: [your_program] TCP Connection 127.0.0.1:38108 established!        
remote: [your_program] Hex string: 1F8B08000000000000034BCA294D4D4A2D2AAA0400D3241E7909000000        
remote: [http-compression-3] Received bytes: "HTTP/1.1 200 OK\r\nContent-Encoding: gzip\r\nContent-Type: text/plain\r\nContent-Length: 29\r\n\r\n\x1f?\b\x00\x00\x00\x00\x00\x00\x03K?)MMJ-*?\x04\x00?$\x1ey\t\x00\x00\x00"
remote: [http-compression-3] < HTTP/1.1 200 OK
remote: [http-compression-3] < Content-Encoding: gzip
remote: [http-compression-3] < Content-Type: text/plain
remote: [http-compression-3] < Content-Length: 29
remote: [http-compression-3] < 
remote: [http-compression-3] < <Binary Content>
remote: [http-compression-3] < 
remote: [http-compression-3] Received response with 200 status code
remote: [http-compression-3] ✓ Content-Encoding header is present
remote: [http-compression-3] ✓ Content-Length header is present
remote: [http-compression-3] Failed to decode gzip: Failed to decompress data: gzip: invalid header
remote: [http-compression-3] Test failed
remote: [http-compression-3] Terminating program
remote: [http-compression-3] Program terminated successfully
remote: 
remote: View our article on debugging test failures: https://codecrafters.io/debug        

```

And here’s a snippet of my code:

```csharp
            var echoMessage = serverRequest.Path.Replace(EchoEndpointPath, string.Empty);

            var acceptEncoding = serverRequest.Headers.Where(x =>
                    string.Equals(x.Key, "Accept-Encoding", StringComparison.InvariantCultureIgnoreCase))
                .Select(x => x.Value)
                .FirstOrDefault()?
                .Split(',')
                .Select(x => x.Trim());

            var containsGzipEncoding = acceptEncoding?
                .Contains("gzip", StringComparer.InvariantCultureIgnoreCase);

            var includeGzip = containsGzipEncoding.HasValue && containsGzipEncoding.Value;

            sb.Append("HTTP/1.1 200 OK\r\n");

            if (includeGzip)
            {
                byte[] compressedData;
                using (var memoryStream = new MemoryStream())
                {
                    using (var gzipStream = new GZipStream(memoryStream, CompressionMode.Compress))
                    {
                        var stringBytes = Encoding.UTF8.GetBytes(echoMessage);
                        gzipStream.Write(stringBytes, 0, stringBytes.Length);
                        gzipStream.Flush();
                    }

                    compressedData = memoryStream.ToArray();

                    Console.WriteLine($"Hex string: {BitConverter.ToString(compressedData).Replace("-", string.Empty)}");
                }
                
                sb.Append("Content-Type: text/plain\r\n");
                sb.Append("Content-Encoding: gzip\r\n");
                sb.Append($"Content-Length: {compressedData.Length}\r\n\r\n");
                sb.Append(Encoding.UTF8.GetString(compressedData));
            }
            else
            {
                sb.Append("Content-Type: text/plain\r\n");
                sb.Append($"Content-Length: {echoMessage.Length}\r\n\r\n");
                sb.Append(echoMessage);
            }

```

---

<div class="post-metadata">

**Author:** ![ValentinJub](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/valentinjub/32/145_2.png) [@ValentinJub](https://forum.codecrafters.io/u/ValentinJub)\
**Post date:** [May 11, 2024, 3:22pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/2 "2024-05-11T15:22:50Z")

</div>

I’m getting the same error in the test, I’ve also tried hex, base64 or binary but they all result in the same error.

```auto
[http-compression-3] $ curl -v -X GET http://localhost:4221/echo/banana -H "Accept-Encoding: gzip"
[http-compression-3] > GET /echo/banana HTTP/1.1
[http-compression-3] > Host: localhost:4221
[http-compression-3] > Accept-Encoding: gzip
[http-compression-3] >
[http-compression-3] Sent bytes: "GET /echo/banana HTTP/1.1\r\nHost: localhost:4221\r\nAccept-Encoding: gzip\r\n\r\n"
[your_program] Parsed & Split Data; [
[your_program] 'GET /echo/banana HTTP/1.1',
[your_program] 'Host: localhost:4221',
[your_program] 'Accept-Encoding: gzip',
[your_program] '',
[your_program] ''
[your_program] ]
[your_program] 11/05/24 15:19:06 Extracted path banana
[your_program] Gzipped data: <Buffer 1f 8b 08 00 00 00 00 00 00 03 4b 4a cc 03 42 00 cf 67 8b 03 06 00 00 00>
[your_program] Gzipped data as a Base64 string: H4sIAAAAAAAAA0tKzANCAM9niwMGAAAA
[your_program] 11/05/24 15:19:06 HTTP/1.1 200 OK
[your_program] Content-Encoding: gzip
[your_program] Content-Type: text/plain
[your_program] Content-Length: 50
[your_program]
[your_program] 1f8b08000000000000034b4acc034200cf678b0306000000
[your_program]
[http-compression-3] Received bytes: "HTTP/1.1 200 OK\r\nContent-Encoding: gzip\r\nContent-Type: text/plain\r\nContent-Length: 50\r\n\r\n1f8b08000000000000034b4acc034200cf678b0306000000\r\n"
[http-compression-3] < HTTP/1.1 200 OK
[http-compression-3] < Content-Encoding: gzip
[http-compression-3] < Content-Type: text/plain
[http-compression-3] < Content-Length: 50
[http-compression-3] <
[http-compression-3] < <Binary Content>
[http-compression-3] <
[http-compression-3] Received response with 200 status code
[http-compression-3] ✓ Content-Encoding header is present
[http-compression-3] ✓ Content-Length header is present
[http-compression-3] Failed to decode gzip: Failed to decompress data: gzip: invalid header
[http-compression-3] Test failed
[http-compression-3] Terminating program
[http-compression-3] Program terminated successfully

```

```auto
function gzipString(inputString) {
    return new Promise((resolve, reject) => {
        // Convert the string to a Buffer since zlib works with binary data
        const buffer = Buffer.from(inputString, 'utf-8');
        zlib.gzip(buffer, (err, result) => {
            if (err) {
                reject(err);
            } else {
                resolve(result);
            }
        });
    });
}

    async _send200(data, content_type, encoding) {
        data = data ? data : '';
        if (encoding[0] === 'gzip') {
            try {
                const gzippedData = await gzipString(data)
                encoding = `Content-Encoding: ${encoding[0]}${CRLF}`
                const encoded = gzippedData.toString('hex') + CRLF
                console.log('Gzipped data:', gzippedData);
                console.log('Gzipped data as a Base64 string:', gzippedData.toString('base64'));
                return `${HTTPResponse[200]}${encoding}Content-Type: ${content_type}${CRLF}Content-Length: ${encoded.length}${CRLF}${CRLF}${encoded}`;
            } catch (e) {
                console.error(e);
                return `${HTTPResponse[500]}Content-Type: text/plain${CRLF}Content-Length: 21${CRLF}${CRLF}Internal Server Error${CRLF}`;
            }
        } else {
            return `${HTTPResponse[200]}Content-Type: ${content_type}${CRLF}Content-Length: ${data.length}${CRLF}${CRLF}${data}${CRLF}`;
        }
    }

```

---

<div class="post-metadata">

**Author:** ![ValentinJub](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/valentinjub/32/145_2.png) [@ValentinJub](https://forum.codecrafters.io/u/ValentinJub)\
**Post date:** [May 11, 2024, 3:46pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/3 "2024-05-11T15:46:59Z")

</div>

Okay I managed to pass the test, my issue was that the gzip encoded data that I was passing wasn’t encoded properly in my response. The test is expecting gzip encoded data that it will open with gunzip and check that it returns the same string in the echo

---

<div class="post-metadata">

**Author:** ![inner](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/inner/32/4695_2.png) [@inner](https://forum.codecrafters.io/u/inner)\
**Post date:** [May 11, 2024, 5:01pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/4 "2024-05-11T17:01:58Z")

</div>

thanks for the hint on gunzip. Tests passed

---

<div class="post-metadata">

**Author:** ![noe-lc](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/noe-lc/32/346_2.png) [@noe-lc](https://forum.codecrafters.io/u/noe-lc)\
**Post date:** [May 14, 2024, 9:18pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/5 "2024-05-14T21:18:16Z")

</div>

Hey! could you please post a snippet of your solution? I’ve already checked that my encoded response indeed returns the same input string after decompression - yet I am still getting the same error…

---

<div class="post-metadata">

**Author:** ![rohitpaulk](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/rohitpaulk/32/6_2.png) [@rohitpaulk](https://forum.codecrafters.io/u/rohitpaulk)\
**Post date:** [May 14, 2024, 9:26pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/6 "2024-05-14T21:26:58Z")

</div>

@noe-lc you’ll find code snippets from users who passed the stage in the “[Code Examples](https://app.codecrafters.io/courses/http-server/stages/http-compression:3/code-examples)” tab:

 ![Screenshot 2024-05-14 at 22.26.36](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/1X/03b94ed9a2054dd6e9836e6193a89ca38ca1d871.png)

---

<div class="post-metadata">

**Author:** ![noe-lc](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/noe-lc/32/346_2.png) [@noe-lc](https://forum.codecrafters.io/u/noe-lc)\
**Post date:** [May 14, 2024, 10:11pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/7 "2024-05-14T22:11:02Z")

</div>

Thanks, I did check the examples - the logic looks is exactly same but just some implementation differences. Still do not know what in my code is wrong ,so I wanted to see how someone with the exact same issue worked around it 👍

---

<div class="post-metadata">

**Author:** ![inner](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/inner/32/4695_2.png) [@inner](https://forum.codecrafters.io/u/inner)\
**Post date:** [May 15, 2024, 2:57am UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/8 "2024-05-15T02:57:55Z")

</div>

In my case, the Encoding.UTF8.GetString(compressedData) was impacting the result as this function call was converting the compatible gzip output to incompatible

---

<div class="post-metadata">

**Author:** ![noe-lc](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/noe-lc/32/346_2.png) [@noe-lc](https://forum.codecrafters.io/u/noe-lc)\
**Post date:** [May 15, 2024, 3:30pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/9 "2024-05-15T15:30:47Z")

</div>

For those stumbling upon this error while using TS and plain socket writes, you need to write the response in two parts:

1. First write the startLine + headers: `socket.write(startLineAndHeaders)`
2. Write the compressed data `socket.write(compressedData)`
3. End the socket with socket.end()

This is done so that the client is prepared to receive gzip encoded data.

---

<div class="post-metadata">

**Author:** ![stevenh-gh](https://yyz1.discourse-cdn.com/flex003/user_avatar/forum.codecrafters.io/stevenh-gh/32/316_2.png) [@stevenh-gh](https://forum.codecrafters.io/u/stevenh-gh)\
**Post date:** [May 20, 2024, 4:15am UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/10 "2024-05-20T04:15:41Z")

</div>

Thanks! This worked for me.  
I was under the impression that I needed to send the hex representation of gzip `zlib.gzipSync(string).toString('hex')` and I would write the entire response with interpolated values. It seems I didn’t need the `toString('hex')` after all I just needed a 2nd write.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/7/0/700657133935c15703e22c7c8870394f6e6dc27d.svg) [@system](https://forum.codecrafters.io/u/system)\
**Post date:** [May 25, 2024, 4:16am UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/11 "2024-05-25T04:16:36Z")

</div>

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex003/uploads/codecrafters/original/3X/7/0/700657133935c15703e22c7c8870394f6e6dc27d.svg) [@system](https://forum.codecrafters.io/u/system)\
**Post date:** [June 3, 2024, 7:51pm UTC](https://forum.codecrafters.io/t/http-server-cr8-gzip-compression/348/12 "2024-06-03T19:51:29Z")

</div>

Note: I’ve updated the title of this post to include the stage ID (#CR8). You can learn about the stages rename here: [Upcoming change: Stages overhaul](https://forum.codecrafters.io/t/upcoming-change-stages-overhaul/375).