Mark a varnish backend as healthy, sick or automatic via CLI

Want to help support this blog? Try out Oh Dear, the best all-in-one monitoring tool for your entire website, co-founded by me (the guy that wrote this blogpost). Start with a 10-day trial, no strings attached.

We offer uptime monitoring, SSL checks, broken links checking, performance & cronjob monitoring, branded status pages & so much more. Try us out today!

Profile image of Mattias Geniar

Mattias Geniar, August 11, 2016

Follow me on Twitter as @mattiasgeniar

This is a useful little command for when you want to perform maintenance on a Varnish installation and want to dynamically mark backends as healthy or sick via the command line, without restarting or reloading varnish.

See varnish backend health status

To see all backends, there are 2 methods: a debug output and a normalized output.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.list
Backend name                   Refs   Admin      Probe
backend1(127.0.0.1,,80)        1      probe      Sick 0/4
fallback(172.16.80.5,,80)      12     probe      Healthy (no probe)

$ varnishadm -S /etc/varnish/secret -T localhost:6082 debug.health
Backend backend1 is Sick
Current states  good:  0 threshold:  2 window:  4
Average responsetime of good probes: 0.000000
Oldest                                                    Newest
================================================================
---------------------------------------------------------------- Happy

The backend.list shows all backends, even those without a probe (= healtcheck) configured.

The debug.health command will show in-depth statistics on the varnish probes that are being executed, including the IPv4 connect state, whether a send/receive has worked and if the response code was HTTP/200.

For instance, a healthy backend will be shown like this, with each state of the check (IPv4, send, receive & HTTP response code) on a seperate line.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 debug.health
Backend backend1 is Healthy
Current states  good:  5 threshold:  4 window:  5
Average responsetime of good probes: 0.014626
Oldest                                                    Newest
================================================================
4444444444444444444444444444444444444444444444444444444444444444 Good IPv4
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX Good Xmit
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR Good Recv
HHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH Happy

Now, to change backend statuses.

Mark a varnish backend as healthy or sick

In order to mark a particular backend as sick or healthy, thus overriding the probe, you can do so like this.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.set_health backend1 healthy

The above command will mark the backend named backend1 as healthy. Likewise, you can mark a backend as sick to prevent it from getting traffic.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.set_health backend1 sick

If you have multiple Varnish backends and they’re configured in a director to load balance traffic, all traffic should gracefully be sent to the other backend(s). (see the examples in mattiasgeniar/varnish-4.0-configuration-templates)

If you mark a backend explicitly as sick, the backend.list output changes and the admin column removes the ‘probe’ and marks it as ‘sick’ explicitly, indicating it was changed via CLI.

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.list
Backend name                   Refs   Admin      Probe
backend1(127.0.0.1,,80)        1      sick       Sick 0/4
fallback(172.16.80.5,,80)      12     probe      Healthy (no probe)

You can also change it back to let Varnish decide the backend health.

Mark the backend as ‘varnish managed’, let probes decide the health

To let Varnish decide the health itself, by using it probes, mark the backend to be auto again:

$ varnishadm -S /etc/varnish/secret -T localhost:6082 backend.set_health backend1 auto

So to summarise: the backend.set_healthy command in varnishadm allows you to manipulate the backend health state of varnish backends, overriding the result of a probe.

Useful when you’re trying to gracefully update several backend servers, by marking backends as sick one by one without waiting for the probes to discover that backends are sick. This method allows you to do things gracefully before the update.



Want to subscribe to the cron.weekly newsletter?

I write a weekly-ish newsletter on Linux, open source & webdevelopment called cron.weekly.

It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.

No spam. Just some good, practical Linux & open source content.