I have been searching for a free open source solution to protect my web application against prying hackers, malicious screen scrapers, illegitimate crawlers, rampant bots and abusive API users. Besides being free and open source, the minimum requirement is that the solution can identify rogue user IP addresses and blacklist them if necessary. Preferably, the solution can also protect (somewhat) against denial-of-service (DOS) attack and implement API rate limiting.
There are a few free solutions available for DOS protection. For example, there is the Apache module mod_evasive. To the best of my knowledge, mod_evasive by itself does not work for a web server located behind a load balancer or proxy because it cannot access the user IP from the X-Forwarded-For header field. The additional installation of mod_rpaf or mod_cloudflare is required to bypass the limitation. At the time of writing, the only way to install both modules on Amazon Linux for Apache 2.2 is to download and compile the source code. Furthermore, mod_evasive is only compatible with Apache running in prefork mode so if your Apache is using MPM worker or event, you are out of luck. To find out which mpm module Apache is using, check the configuration file at
The alternative solution I have been exploring is ModSecurity. To install, run
sudo yum install mod_security OR sudo yum install mod24_security
ModSecurity is a web application firewall (WAF) designed to protect Apache, NGINX and IIS against common hacking exploits. It works by examining web requests against a set of rules to identify malicious traffic pattern (e.g. HTTP header missing user-agent) and execute the corresponding actions (e.g. drop connection). To make life easier, you can download a predefined set of generic attack detection rules called the OWASP ModSecurity Core Rule Set (CRS) via
sudo yum install mod_security_crs
You can take a look at what the rules look like at https://github.com/SpiderLabs/owasp-modsecurity-crs
The CRS rules are installed at
You may also add your own rules at
Out of the box, the CRS rules will likely generate many false alarms for your particular website. This means it will inadvertently shut your users off from your site if you are not careful. For example, it may mistakenly identify a legitimate HTTP POST request with more than 255 parameters as an exploit even if your application expects it.
At the minimum, before you deploy ModSecurity to production use, find the following line from ModSecurity configuration file at
and change it to:
This sets ModSecurity to detection mode so it only reports potential exploits without enforcement. Every time you make changes to the configuration or rules, you must restart Apache with sudo service httpd restart. If everything goes well, your web application should function normally as before while ModSecurity checks every web requests and log potential problems to
You can control what information ModSecurity should log by editing the configuration file at
You can tailor the burst detection pattern by editing the file at
# # -- [[ DoS Protection ]] ---------------------------------------------------------------- # # If you are using the DoS Protection rule set, then uncomment the following # lines and set the following variables: # - Burst Time Slice Interval: time interval window to monitor for bursts # - Request Threshold: request # threshold to trigger a burst # - Block Period: temporary block timeout # SecAction \ "id:'900015', \ phase:1, \ t:none, \ setvar:'tx.dos_burst_time_slice=60', \ setvar:'tx.dos_counter_threshold=100', \ setvar:'tx.dos_block_timeout=600', \ nolog, \ pass"
ModSecurity is a powerful tool to protect web applications and as such it comes with a learning curve. I have only touched on the basics in this blog entry. Hopefully, I can devote some more blog time to it as I pick up the tool myself.
To develop your own tunnel vision quickly as I have been doing, I recommend taking a look at the official documentation at:
- Install dependencies
sudo yum install pcre-devel sudo yum install gcc make sudo yum install libxml2 libxml2-devel httpd-devel pcre-devel curl-devel
- Installing Mod_Evasive
cd /usr/src sudo wget http://www.zdziarski.com/blog/wp-content/uploads/2010/02/mod_evasive_1.10.1.tar.gz sudo tar xzf mod_evasive_1.10.1.tar.gz cd mod_evasive sudo apxs -cia mod_evasive20.c
- Configuring Mod_Evasive
By default installation adds the following line of mod_evasive configuration to your Apache configuration file. Please verify that it should be there like similar to below. If you can’t see this below line, then add this to your httpd.conf file.
sudo nano /etc/httpd/conf/httpd.conf LoadModule evasive20_module /usr/lib64/httpd/modules/mod_evasive20.so
Now add the mod_evasive configuration parameters to your Apache configuration at the end. Replace email@example.com with your Email Id to get email alerts.
<IfModule mod_evasive20.c> DOSHashTableSize 3097 DOSPageCount 2 DOSSiteCount 50 DOSPageInterval 1 DOSSiteInterval 1 DOSBlockingPeriod 60 DOSEmailNotify firstname.lastname@example.org </IfModule>
Next restart the Apache service to update changes.
sudo nano service httpd restart
- Installing Mod_Cloudflare
sudo yum groupinstall "Development tools" sudo yum install httpd24-devel sudo wget https://raw.githubusercontent.com/cloudflare/mod_cloudflare/master/mod_cloudflare.c sudo apxs -a -i -c mod_cloudflare.c