Nginx Proxy: upstream sent too big header while reading response header from upstream

Mattias Geniar, Friday, March 11, 2016

If you're running an Nginx proxy configuration, you can run into the following error in your logs.

$ tail -f error.log
[error] 19896#0: upstream sent too big header while reading response header from upstream,
   client: 10.5.120.6,
   server: host.tld,
   request: "GET /path/to/file.php HTTP/1.1",
   upstream: "",
   host: "host.tld"

[error] 19899#0: upstream sent too big header while reading response header from upstream,
   client: 10.5.120.6,
   server: host.tld,
   request: "GET /path/to/file.php HTTP/1.1",
   upstream: "",
   host: "host.tld"

In your Nginx proxy configuration, you probably have something very similar to this:

server {
  listen        80;
  server_name   host.tld;

  location / {
    proxy_pass       http://upstream;
    ...
  }
}

To resolve this issue, increase the proxy buffers that nginx uses. Before nginx sends a response back to your visitor, it will cache the request it had to make from its upstream. However, there are limited buffers available to buffer such a response. If the HTTP headers contain more info than anticipated, those proxy buffers can get saturated and nginx will drop the request with the "upstream sent too big header while reading response header from upstream" error in your logs.

To resolve it, add the proxy_buffer_size configurations to your location block.

server {
  listen        80;
  server_name   host.tld;

  location / {
    proxy_pass       http://upstream;
    ...

    proxy_buffer_size          128k;
    proxy_buffers              4 256k;
    proxy_busy_buffers_size    256k;
  }
}

That change increases Nginx's proxy buffer from the default 4KB to 128KB, usually big enough to cache any backend response without posing a risk to your server. After all, that server now has to assign more memory to buffer each backend response.



Hi! My name is Mattias Geniar. 👋 I'm an independent software developer ⌨️ & Linux sysadmin 👨‍💻, a general web geek & public speaker. Currently working on DNS Spy & Oh Dear! Follow me on Twitter as @mattiasgeniar 🐦.

🔥 If you're stuck with a technical problem, I'm available for hire to help you fix it!

Share this post

Did you like this post? Help me share it on social media! Thanks. 🤗

Have feedback?

New comments have been disabled on this blog, existing comments will remain as-is. Want to give feedback? Is there a mistake in the post?

Send me a tweet on @mattiasgeniar!

Comments

rajeev Thursday, August 2, 2018 at 08:23 -

followed your guide and tried 10-15 time to change buffer size , after saving the file when i try to run reaction website got the same error 502 bad gateway . and below line of code has been also removed
” proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;”

is that file creating automatically ?? how can i resolve it ?


Ben Abrams Friday, September 28, 2018 at 21:54 -

> That change increases Nginx’s proxy buffer from the default 4KB to 128KB, usually big enough to cache any backend response without posing a risk to your server. After all, that server now has to assign more memory to buffer each backend response.

These settings are per connection (it’s not a pool), depending on what you in terms of resources and number of connections those numbers at best are wasteful and at worst could leave you (or your customers) to unintentionally `DOS`ing your server when you create too many connections at once as you will exhaust your memory. If you don’t want to do the math to figure out what you want to support then I would recommend doubling it rather than choosing such large numbers. Also something to keep in mind is that if you are putting that much data into your headers you _may_ be doing something wrong. A real example I just saw from a developer at my current employer was shoving the entire url into the cookie which is not needed and is wasteful.


Name Saturday, May 11, 2019 at 01:59 -

^^ That guy (Ben Abrams) is right.


Mr. L Saturday, June 8, 2019 at 23:58 -

Thank you! Thank you! Thank you!!!!!!!!!!


Ionel Hales Friday, October 11, 2019 at 16:36 -

I faced the same error message but above solution didn’t work for me. After more research I found the following settings:

client_header_buffer_size;
large_client_header_buffers;

which worked for me.
Full example:
https://www.nginx.com/resources/wiki/start/topics/examples/fullexample2/


Inbound links