E-Scribe : a programmer’s blog

About Me

PBX I'm Paul Bissex. I build web applications using open source software, especially Django. Started my career doing graphic design for newspapers and magazines in the '90s. Then wrote tech commentary and reviews for Wired, Salon, Chicago Tribune, and others you never heard of. Then I built operations software at a photography school. Then I helped big media serve 40 million pages a day. Then I worked on a translation services API doing millions of dollars of business. Now I'm building the core platform of a global startup accelerator. Feel free to email me.

Book

I co-wrote "Python Web Development with Django". It was the first book to cover the long-awaited Django 1.0. Published by Addison-Wesley and still in print!

Colophon

Built using Django, served with gunicorn and nginx. The database is SQLite. Hosted on a FreeBSD VPS at Johncompanies.com. Comment-spam protection by Akismet.

Pile o'Tags

Stuff I Use

bitbucket, Django, Emacs, FreeBSD, Git, jQuery, LaunchBar, Markdown, Mercurial, OS X, Python, Review Board, S3, SQLite, Sublime Text, Ubuntu Linux

Spam Report

At least 236429 pieces of comment spam killed since 2008, mostly via Akismet.

Protecting the server with mod_evasive

As I hinted in my report several days ago about this server suffering a DOS attack, I've taken some measures to prevent a repeat occurrence. One of them was to install the mod_evasive Apache module, which was suggested by a number of people.

(There's also mod_security. It's way more complex than I need right now, but would be worth looking at for a busier server.)

Initially I was skeptical, since mod_evasive doesn't seem to be a very actively maintained project. But it looked so simple I decided to give it a try. Installing via FreeBSD ports was, as usual, nearly instantaneous. Configuration looks like this (I've omitted a few settings for simplicity's sake, and these are not the actual numbers I'm using):

<IfModule mod_evasive20.c>
    DOSPageCount        2
    DOSPageInterval     1
    DOSSiteCount        50
    DOSSiteInterval     1
    DOSBlockingPeriod   10
</IfModule>

Simply put: "If any client makes more than two requests for a single URL per second, blacklist them for 10 seconds; if any client makes more than 50 requests total per second, blacklist them for 10 seconds."

Additional requests during the blacklist period add another 10 seconds. DOSSiteCount is higher to accommodate media files; loading this single page has caused your browser to make seven separate requests to my server, for example, and many pages are more complex than that. But nobody has a legitimate reason to make multiple requests for the same resource more than once per second. Not with my sites, anyway.

When blacklisting is triggered, mod_evasive can send a notification email, write to a logfile, and/or pass the IP to a script for further processing (e.g adding the IP to firewall rules).

In the past four days, mod_evasive has blocked 42 IPs. Using my elite Unix shell skills I got the hostnames:

cat ips.txt | sort | xargs -L 1 host

The main thing I was looking for, and thankfully didn't find, was search engine spiders -- the last thing I want to do is block Google from crawling my sites. Otherwise, no really interesting patterns emerged. About half of the addresses were broadband connections, likely zombies.

One thing I like about having this measure in place is that it doesn't privilege one form of stupidity over another: badly-written web crawlers, hyperactive blogspam scripts, and actual deliberate DOS attacks all get the same treatment.

Wednesday, April 25th, 2007
+ + +
5 comments

Comment from Michael , 1 week later

It sounds good that it is possible to protect the server from a DoS. But waht can you do if you have only rented some webspace at your hosting provider and not a whole server? Is there a similar tool available or does hosting providers normally install mod_evasive to avoid DoS?

Comment from Paul , 1 week later

Michael, in that situation you're really at the mercy of your hosting provider. Usually such rules are added at the firewall level, not at the webserver level. But in any case, there's not a whole lot you can do as a user on a shared box. At best you might be able to write some .htaccess rules or maintain a blacklist at the application level, but that's no fun. If you're concerned, I'd say ask your hosting provider what they offer, and if you're not satisfied with the answer then it's time to go shopping.

Comment from Libby , 8 weeks later

I'm curious. We installed mod_evasive onto our red hat server last february. It's been working beautifully. However recently our server has been going down due to DoS attacks at a rate of once a month for about five minutes. I haven't found any new patches that we may be missing. Any thoughts on this?

Comment from Paul , 8 weeks later

Hi Libby -- If what you're saying is that you're seeing attacks that mod_evasive should be catching, I'd probably start taking a look at mod_security. Otherwise, not sure what to suggest besides the obvious tack of lowering mod_evasive tolerances. Good luck!

Comment from brixton , 3 years later

The above link for the mod_evasive web site is incorrect. It is located here (as of current): http://www.zdziarski.com/blog/?page_id=442

Comments are closed for this post. But I welcome questions/comments via email or Twitter.