Nginx + Fail2ban Blocking IP behind AWS Load Balancer

You know there is this problem, if you have some web servers behind an Amazon ELB it’s hard to block some malicious hosts. Why is it hard? Well, first of all, on the ELB you can use only security groups to allow/block IP/nets, BUT AWS Security Group doesn’t permit you to block something but only to allow, so it’s impossible blocking an IP at the source (ELB).

Wait, I mean, it could be possible while you add an instance as Firewall before the ELB, but this is going to be complex to add and manage and it could be hard to find the ip to ban because you haven’t the nginx logs because the host doesn’t reach it yet. So you should bring up an Nginx on the Firewall to proxy every requests to the ELB and then you could use Fail2ban directly there.

But again, this isn’t a fast solution and I think you could like this one much more.

Forget firewalling the ELB and concentrate on the web servers (normally in autoscaling if behind an ELB). Install fail2ban on each web server (or on the AMI to have it in each autosclale server).

When you have fail2ban installed, configure it to catch and ban your malicious hosts, you can find useful information how to block hosts with fail2ban in this post I wrote months ago.

Now, if you try to start it and wait the attackers coming, you should find blocked IP with “iptables -L“, but:

“hey WTF! The ip is blocked but I’m still receiving a lot of packets from that host!”

Yes dude, this happens because fail2ban block the host IP but you are receiving the packets from the ELB IP, so blocking the real host IP is completely useless.

So what can I do? Here is the cool part, you can’t still block/drop/reject those packets but you can deny the IP to do the attack simply returning it a 403 Forbidden for each url it tries.

First of all you need to add these two lines to your Nginx.conf on the http section:

 real_ip_header X-Forwarded-For; set_real_ip_from;

This will tell Nginx to use the real host IP instead the proxied from the ELB so you can block that instead blocking all the ELB (that’s not good at all…)

Then use a simple script to automatically add/remove the deny lines you need, I did this:

1- On my virtual host config I added an include like this:

location / { include /etc/nginx/conf.d/deny-hosts; }

2- On my fail2ban default action I added an external script to write and remove the lines needed to deny-hosts file. Find the file /etc/fail2ban/action.d/iptables-multiport.conf and add the lines to ban/unban actions

actionban = iptables -I fail2ban- 1 -s -j DROP /etc/fail2ban/ ban actionunban = iptables -D fail2ban- -s -j DROP /etc/fail2ban/ unban

And here is the simple script:

#!/bin/bash if [ $# -eq 0 ]  then  echo "No arguments supplied"  exit fi # In any case remove the ip this is useful to prevent dups sed -i "/deny $2;/d" /etc/nginx/conf.d/deny-hosts # If action is ban add ip to deny-hosts file if [ $1 == "ban" ] then  echo "deny $2;" >> /etc/nginx/conf.d/deny-hosts fi # Reload Nginx Config to get banned IP /usr/sbin/service nginx reload

Now, every time fail2ban will find an IP to ban it will use the script to add the host to be banned and reload nginx. The host will receive a 403 Forbidden for each URL it tries untill the unban action start removing the deny lines and reloading Nginx.

Hope you find this useful!

Tags: amazon, attack, aws, block, elb, fail2ban, hackers, hacks, internet, linux, nginx, script, script code

Post simili:

If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.