I first mentioned Slowloris on this blog in 2009, more than 6 years ago. To this day, it’s still a very effective attack on Apache servers.
How it works
Slowloris holds connections open by sending partial HTTP requests. It continues to send subsequent headers at regular intervals to keep the sockets from closing. It’s a SYN-flood attack, but aimed directly at Apache.
This is particularly nasty, because it won’t show up in your webserver logs until a request has finished, and it’s the design of Slowloris to never finish requests and just keep them open.
You won’t detect slowloris in your logs, you have to use other tools to detect such an attack.
Starting a slowloris attack on Apache
Slowloris is a perl script, you can grab it from my mirrored github repo. Download the perl script and execute it.
$ ./slowloris.pl -dns your.target.tld -port 80 -timeout 2000 -num 750
The above will connect to your.target.dl on port 80 and attempt to make 750 connections to Apache and keep them open.
What it looks like on the server
To be on the receiving end of a Slowloris attack, you’ll see the following.
If your apachectl status still works (it probably won’t, because all your httpd processes will be busy), it will look like this.
$ apachectl status ... CPU Usage: u2.18 s.2 cu0 cs0 - .27% CPU load .817 requests/sec - 11.1 kB/second - 13.5 kB/request 131 requests currently being processed, 2 idle workers RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR RRRRRRRRRRRRRRRRRRRRRRRRRRWRRRRRRRKRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
The symptoms are: very low CPU usage, a lot of Apache processes, very few new requests/s.
$ ps faux | grep httpd | wc -l 113
Slowloris works by making more and more requests, until it reaches your Apache’s MaxClients limit.
In Apache 2.4, it looks like this.
$ tail -f /var/log/httpd/error.log ... [mpm_prefork:error] [pid 7724] AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
For Apache 2.2, it looks like this.
$ tail -f /var/log/httpd/error.log ... [error] server reached MaxClients setting, consider raising the MaxClients setting
The symptoms are always the same: MaxClients will be reached. It’s how Slowloris prevents new connections from coming through.
What it looks like for a visitor of your site
Anyone trying to connect to your site, will have a “connecting” icon that keeps waiting forever.
The site won’t load and your visitors will never get to see the content. If you have a webshop, you’ll miss sales.
Identify the attacking IP address
You can detect the attack if you see such logs, but you don’t know who started the attack: the source IP isn’t logged until the HTTP requests are finished.
You can use netstat to list the most active IPs on your server. Slowloris may try to hide from the Apache service, but it can’t hide from the network. Every request to Apache is still a connection to the server.
$ netstat -ntu -4 -6 | awk '/^tcp/{ print $5 }' | sed -r 's/:[0-9]+$//' | sort | uniq -c | sort -n
The above will filter all IPs from your current server, order them and then count each unique occurence. You’ll find the most often connected IPs at the bottom.
If you’re running an old version of netstat (like CentOS 5.x versions), you may get an error like “netstat: invalid option -– 4". In that case, go for the following altered one-liner.
$ netstat -ntu | awk '/^tcp/{ print $5 }' | sed -r 's/:[0-9]+$//' | sort | uniq -c | sort -n
The output looks like this.
$ netstat -ntu -4 -6 | awk '/^tcp/{ print $5 }' | sed -r 's/:[0-9]+$//' | sort | uniq -c | sort -n
...
40 186.2.xx.xx
44 127.0.0.1
105 92.243.xx.xx
I had 105 connections from 92.243.xx.xx, which isn’t normal for a webserver. Chances are, that’s the one performing the Slowloris attack.
Blocking a Slowloris attack by blocking the IP
Once you’ve identified the IP, block it on your server. There’s multiple ways to block an IP, like iptables, route, ip, … I prefer the simple ip add syntax to blackhole an IP.
$ ip route add blackhole 92.243.xx.xx
Restart your Apache server, to clear all connections, and you should be good – until the attacker switches IP.
$ service httpd restart $ systemctl restart httpd
All systems go.
Preventing Slowloris attacks
Slowloris abuses a fundamental design flaw in the Event MPM of Apache. Each connection gets a thread. There isn’t much to do about that.
To effectively prevent Slowloris, your best bet is to enable some kind of proxy between the client and your Apache webserver. Tools like Varnish, Nginx or HAProxy are perfect for this. Their server design is different and can handle a lot more connections than Apache.
If you have to stick to Apache, there are few choices for you:
- Enable rate-limiting from a single IP in iptables
- Test the mod_antiloris or mod_evasive Apache module
To limit connections to port :80 from a single IP, use the following iptables rule.
$ iptables -I INPUT -p tcp --dport 80 -m connlimit --connlimit-above 50 --connlimit-mask 20 -j DROP
The --connlimit-above 50 will allow at most 50 connections. The --connlimit-mask 20 groups IPs using that prefix length. Every IP from the same /20 network is subject to that 50 connection limit.
Tune those numbers as you see fit, increase the connection limits or decrease the prefix.