Posts tagged: WEBSERVERS

A weather site in six (declarative) lines

For several years I maintained a hodge-podge of little web-based utilities at toolbot.com. Recently I decided to wipe the slate clean, bringing things back selectively. One of the few that I missed personally was my weather site. It returned National Weather Service forecasts in response to compact URLs of the form weather.toolbot.com/05667, for any five-digit US zipcode.

Originally, the site worked by scraping plaintext forecast files on the NWS servers. Eventually those went away; I looked at and gave up on the arcane (to me) SOAP interface that superseded them. All I wanted was a damn weather forecast, without ads, via an easy-to-type-no-matter-where-I-am URL.

Movin' on up

My last post was about a server migration at work; this one’s about moving my own stuff, an operation completed late last night. It was easier in that it’s less critical (nobody’s likely to freak out if my blog disappears for a day) but harder in that it also involved mail. I hate dealing with mail; though I did manage the switch without losing any, as far as I know. My thanks to the JohnCompanies.com staff for helping this go smoothly.

Root, sweet root

For several weeks at work I’ve been prepping for a server move; this week we flipped the switch. It was the most serious migration I’ve ever done, and it went very well. Some notes:

  • Previously we shared a dedicated box at a certain very large colo provider. A few weeks ago, in the course of swapping out a failing drive in our box, staffers at the colo 1) wiped out the live backup of the drive and then 2) destroyed the contents of the failing drive, overwriting it with a week-old backup. A week is a lot for us, with a couple limited-access applications (Django apps, naturally) seeing steady daily use by hundreds of users. This just firmed my resolve to go with a smaller provider (JohnCompanies) who has taken good care of me over the past three years.

  • The other thing the above incident drove home is that backups are good. We lost no data in the end, but recovering some of it was a real pain (in one case reconstructing database inserts from logfiles). So, the new server, in addition to being backed up daily onsite, backs up daily to an rsync.net server. There are also hourly and daily cronjobs making database dumps – at any given point I have a snapshot for each of the previous 24 hours and one for each of the previous 7 days. And finally, we can pull down local copies from the rsync.net server. It’s about a gig of data right now; I burned it to two CDs and brought them home for safekeeping.

  • There are three Django projects running on the server, and more to come. All back-end content administration is handled by Django, and in the fall I plan to replace the remaining front-end PHP pieces. Having full control over the box and Apache makes my Django work much easier. Perhaps oddly, I have three different versions of the source running in parallel (0.91, pre-merge magic-removal, and current trunk), roughly reflecting the times when the three projects were created. These are all straight Subversion checkouts living in /usr/local/www/django, and I set the PYTHONPATH appropriately for each in my Apache config.

  • I had tested most of our software configuration on a play server, so setup was fairly straightforward. The glitches were minor and familiar: an “HTTP Accept” filter warning on Apache startup, a thread-related workaround for Apache, and a stable SVN checkout of mod_python (rather than the current 3.2.8 release) needed to work with Apache 2.2.

  • I’m using portsnap instead of CVSup, and ssmtp instead of sendmail.

  • The fact that our email is handled by a separate provider (not my department, thankfully) made this much easier than it otherwise would have been.

  • The shared colo box was well run – but there’s no substitute for having root!

Anatomy of a BoingBoinging

Anatomy of a BoingBoinging

spike Some domain names become active verbs: I googled it. Others become passive verbs: I got slashdotted.

BoingBoing, linked to by over 16,000 blogs, is a passive verb too, and two weeks ago my server got BoingBoinged.

Joe started it when he made a posting on the Well with a link to a series of (bloodless) photos from a huge motorcycle ride turned motorcycle pileup. Somebody suggested a slideshow; I took the opportunity to exercise my mass-image-resize script and to check out ImageReady’s ability to export animations as Flash. I put the resulting 2.6MB file on my neglected moto-blog, posted the link to the Well, and went on with my evening.

thttpd

Though LightTPD (or “Lighty”) is the darling lightweight webserver of 2005, Jef Poskanzer’s thttpd has been serving static content securely and wickedly fast since 1995 or so.

O’Reilly’s ONLamp.com recently posted a tutorial, “Lightweight Web Serving with thttpd”, which is worth a look if you run a high-traffic site or have underpowered server hardware. The article walks you through installing and configuring thttpd, setting up CGI and virtual host support, and using thttpd as a static content server behind a more full-featured server like Apache.