Test Gzip Compression of Site via Curl

Want to help support this blog? Try out Oh Dear, the best all-in-one monitoring tool for your entire website, co-founded by me (the guy that wrote this blogpost). Start with a 10-day trial, no strings attached.

We offer uptime monitoring, SSL checks, broken links checking, performance & cronjob monitoring, branded status pages & so much more. Try us out today!

Profile image of Mattias Geniar

Mattias Geniar, November 27, 2014

Follow me on Twitter as @mattiasgeniar

Curl is, without a doubt, one of the most powerful command-line tools for testing and debugging websites. It allows you to customise your HTTP request in any way you desire.

To test if a website is performing Gzip compression, you can of course use curl. But just a “curl” call will not work, since you have to add additional headers to indicate you are capable of handling a gzip’d response – where curl would, by default, expect a normal plain text response.

A normal curl request can look like this. Note I’m not using the -I (capital i) parameter, as that sends a HEAD instead of a GET request, which may cause different code to be executed on the server (this matters when working with picky HTTP servers or REST API’s that use different methods for different actions).

$ curl -i https://ma.ttias.be 2>/dev/null | head -n 13
HTTP/1.1 200 OK
Server: nginx
Date: Thu, 27 Nov 2014 22:12:59 GMT
Content-Type: text/html
Content-Length: 44481
Last-Modified: Thu, 27 Nov 2014 22:12:31 GMT
Connection: keep-alive
Vary: Accept-Encoding, Cookie
Accept-Ranges: bytes

The response HTTP headers can be checked with the -i (lower case i) command. It performs the GET request as you would expected, but shows you the response headers as well. Since I don’t care about the entire response, I just trim the first few lines. However, the output doesn’t show a Content-Encoding header, so the output is plain text – no compression. This is clear from the output as well, if you would leave out the head pipe, you would see the HTML in plain text lower in the output.

To test for gzip, you have to tell the server you’re capable of processing a compressed response, with the Accept-Encoding header.

$ curl -H "Accept-Encoding: gzip" -i https://ma.ttias.be 2>/dev/null | head -n 13
HTTP/1.1 200 OK
Server: nginx
...
Content-Encoding: gzip
Accept-Ranges: bytes

The result is a clear indication of a compressed result, in the Content-Encoding: gzip HTTP header. You’ll also notice that the output of the page is no longer human readable, as it’s a compressed page that is being sent. The see the output, you would have to gunzip it.

Alternative compression algoritmes/methods are compress (used by mod_deflate in Apache) or SDCH (Google Shared Dictionary Compression Over HTTP), used in Chrome. Most browsers send a combination, so the server can pick the algoritme/method it prefers.

$ curl -H "Accept-Encoding: gzip, deflate, sdch" -i https://ma.ttias.be 2>/dev/null | head -n 13
HTTP/1.1 200 OK
Server: nginx
...
Content-Encoding: gzip
Accept-Ranges: bytes

In the above example, from the 3 accepted encodings (gzip, deflate or sdch), my server preferred gzip.



Want to subscribe to the cron.weekly newsletter?

I write a weekly-ish newsletter on Linux, open source & webdevelopment called cron.weekly.

It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.

No spam. Just some good, practical Linux & open source content.