Nginx: add_header not working on 404

Oh Dear! monitors your entire site, not just the homepage. We crawl and search for broken pages and mixed content, send alerts when your site is down and notify you on expiring SSL certificates.

Start your free 10 day trial! »

Image of Mattias Geniar

Mattias Geniar, June 07, 2019

Follow me on Twitter as @mattiasgeniar

I had the following configuration block on an Nginx proxy.

server {
  listen        80 default;
  add_header X-Robots-Tag  noindex;
  ...
}

The idea was to add the X-Robots-Tag header on all requests being served. However, it didn’t work for 404 hits.

$ curl -I "http://domain.tld/does/not/exist"
HTTP/1.1 404 Not Found
...

… but no add_header was triggered.

Turns out, you need to specify the header should be added always, not just on successfull requests.

server {
  listen        80 default;
  ...

  add_header X-Robots-Tag  noindex always;
  ...
}

This fixes it, by adding always to the end of the add_header directive.



Want to subscribe to the cron.weekly newsletter?

I write a weekly-ish newsletter on Linux, open source & webdevelopment called cron.weekly.

It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.

No spam. Just some good, practical Linux & open source content.