E-Scribe News : a programmer’s blog

About Me

PBX I'm Paul Bissex, and e-scribe.com is my consulting business. I build web applications using open source software, especially Django. In the '90s I did graphic design for newspapers and magazines. Then I wrote technology commentary and reviews for Wired, Salon.com, Chicago Tribune, and lots of little places you've never heard of. Feel free to email me.

Book

I'm co-author of "Python Web Development with Django", an excellent guide to my favorite web framework. Published by Addison-Wesley, it is available from Amazon and your favorite technical bookstore as well.

Colophon

Built using Django, served by Apache and mod_wsgi. The database is SQLite. The operating system is FreeBSD, on a VPS hosted at Johncompanies.com. Comment-spam protection by Akismet. Vintage topo imagery from the Maptech archive. The markup engine is Markdown.

Pile o'Tags

ajax apple blogs browsers business design django ebay fp fun google hardware haskell intellectual property iphone javascript linux media mercurial mobile open source os x oscon oscon2007 palm php programming python rails rants reading risks ruby site spam textmate the well tips tools unix vcs web development web frameworks webservers wmassdevs writing

Stuff I Use

Akismet, bitbucket, del.icio.us, Django, Emacs, FreeBSD, Git, jQuery, LaunchBar, Markdown, Mercurial, OS X, Postfix, Python, Review Board, S3, SQLite, TextMate, Ubuntu Linux

Spam Report

At least 95836 pieces of comment spam killed since January 2008, mostly via Akismet.

Protecting the server with mod_evasive

As I hinted in my report several days ago about this server suffering a DOS attack, I've taken some measures to prevent a repeat occurrence. One of them was to install the mod_evasive Apache module, which was suggested by a number of people.

(There's also mod_security. It's way more complex than I need right now, but would be worth looking at for a busier server.)

Initially I was skeptical, since mod_evasive doesn't seem to be a very actively maintained project. But it looked so simple I decided to give it a try. Installing via FreeBSD ports was, as usual, nearly instantaneous. Configuration looks like this (I've omitted a few settings for simplicity's sake, and these are not the actual numbers I'm using):

<IfModule mod_evasive20.c>
    DOSPageCount        2
    DOSPageInterval     1
    DOSSiteCount        50
    DOSSiteInterval     1
    DOSBlockingPeriod   10
</IfModule>

Simply put: "If any client makes more than two requests for a single URL per second, blacklist them for 10 seconds; if any client makes more than 50 requests total per second, blacklist them for 10 seconds."

Additional requests during the blacklist period add another 10 seconds. DOSSiteCount is higher to accommodate media files; loading this single page has caused your browser to make seven separate requests to my server, for example, and many pages are more complex than that. But nobody has a legitimate reason to make multiple requests for the same resource more than once per second. Not with my sites, anyway.

When blacklisting is triggered, mod_evasive can send a notification email, write to a logfile, and/or pass the IP to a script for further processing (e.g adding the IP to firewall rules).

In the past four days, mod_evasive has blocked 42 IPs. Using my elite Unix shell skills I got the hostnames:

cat ips.txt | sort | xargs -L 1 host

The main thing I was looking for, and thankfully didn't find, was search engine spiders -- the last thing I want to do is block Google from crawling my sites. Otherwise, no really interesting patterns emerged. About half of the addresses were broadband connections, likely zombies.

One thing I like about having this measure in place is that it doesn't privilege one form of stupidity over another: badly-written web crawlers, hyperactive blogspam scripts, and actual deliberate DOS attacks all get the same treatment.

Wednesday, April 25th, 2007
+ + +
5 comments

Some related posts: Library of 1000 scammy spams • Form hijacking
• Comment from Michael , 1 week later

It sounds good that it is possible to protect the server from a DoS. But waht can you do if you have only rented some webspace at your hosting provider and not a whole server? Is there a similar tool available or does hosting providers normally install mod_evasive to avoid DoS?

• Comment from Paul , 1 week later

Michael, in that situation you're really at the mercy of your hosting provider. Usually such rules are added at the firewall level, not at the webserver level. But in any case, there's not a whole lot you can do as a user on a shared box. At best you might be able to write some .htaccess rules or maintain a blacklist at the application level, but that's no fun. If you're concerned, I'd say ask your hosting provider what they offer, and if you're not satisfied with the answer then it's time to go shopping.

• Comment from Libby , 8 weeks later

I'm curious. We installed mod_evasive onto our red hat server last february. It's been working beautifully. However recently our server has been going down due to DoS attacks at a rate of once a month for about five minutes. I haven't found any new patches that we may be missing. Any thoughts on this?

• Comment from Paul , 8 weeks later

Hi Libby -- If what you're saying is that you're seeing attacks that mod_evasive *should* be catching, I'd probably start taking a look at mod_security. Otherwise, not sure what to suggest besides the obvious tack of lowering mod_evasive tolerances. Good luck!

• Comment from brixton , 3 years later

The above link for the mod_evasive web site is incorrect. It is located here (as of current): http://www.zdziarski.com/blog/?page_id=442

Post a comment

Thanks for reading! Please note: Your comment will not appear until approved, which may take a few hours or more. Spammers will be torpedoed.


(Will not be shared)

(Optional)