Kurt McKee

lessons learned in production

Enabling Brotli compression

Posted 2 January 2020 in website

Have you ever heard of Brotli compression? Until recently I was unaware that it existed.

As it turns out, Google created an efficient new compression algorithm named Brotli back in 2013 and it's been added to most major browsers. It's great for text content like HTML and CSS, and I've enabled it for my site. If you're using a modern browser this content is coming to you courtesy of Brotli compression!

    An image of Patrick Stewart as Star Trek captain Jean-Luc Picard
    with this quote: "I look forward to your report Mr. Brotli." It is
    a parody of a line he delivers in the episode "Hollow Pursuits".

I compiled a dynamic nginx module (which means that it has to be specified in the configuration file), and since this is currently a static-only website I chose to compile only the static Brotli support...making it a dynamic static module. Here are the rough steps that I took to compile it:

# Determine the installed nginx version.
nginx -V

# Download the exact nginx version's source code.
curl https://nginx.org/download/nginx-{VERSION}.tar.gz --output nginx-{VERSION}.tar.gz
tar -xzf nginx-{VERSION}.tar.gz

# Download the Brotli nginx module source code.
git clone https://github.com/google/ngx_brotli.git
cd ngx_brotli/
git submodule update --init
cd ..

# I needed to install these to compile the module.
sudo apt install libpcre3-dev libssl-dev

# Compile the Brotli module with switches that exactly
# match the installed nginx version's switches. I didn't
# automate this yet so you'll have to copy-and-paste.
nginx -V
./configure {LONG LINE OF SWITCHES} --add-dynamic-module=../ngx_brotli/static/
make modules
sudo cp objs/ngx_http_brotli_static_module.so /usr/lib/nginx/modules/

Then, I wrote a Pelican plugin that pre-compresses all of my files. I'll release the plugin soon but it needs some work. For example, I want the plugin to handle all of my compression needs (including gzip), not just Brotli.

UPDATE: I've released the plugin!

Oh, and I also enabled "Vary" headers, since that apparently matters to buggy caching proxies. Here's a fragment of my nginx configuration file:

load_module ngx_http_brotli_static_module.so;

http {
    brotli_static on;
    gzip_static on;
    gzip_vary on;

Note that I haven't enabled dynamic Brotli or gzip compression since everything on the site is currently just a static file anyway.