I had the following configuration block on an Nginx proxy.
server {
listen 80 default;
add_header X-Robots-Tag noindex;
...
}
The idea was to add the X-Robots-Tag header on all requests being served. However, it didn’t work for 404 hits.
$ curl -I "http://domain.tld/does/not/exist" HTTP/1.1 404 Not Found ...
… but no add_header was triggered.
Turns out, you need to specify the header should be added always, not just on successfull requests.
server {
listen 80 default;
...
add_header X-Robots-Tag noindex always;
...
}
This fixes it, by adding always to the end of the add_header directive.