E-Scribe News : a programmer’s blog

About Me

PBX I'm Paul Bissex, and e-scribe.com is my consulting business. I build web applications using open source software, especially Django. I teach photographers web design and professional skills. In the '90s I did graphic design for newspapers and magazines. Then I wrote technology commentary and reviews for Wired, Salon.com, Chicago Tribune, and lots of little places you've never heard of. Feel free to email me.

Book

Python Web Development with Django I'm co-author of "Python Web Development with Django", an excellent guide to my favorite web framework. Its strong points include an introduction to Python, and better coverage of Django 1.0 than nearly anybody else. Published by Addison-Wesley, it is available from Amazon and your favorite technical bookstore as well.

Colophon

Built using Django, served by Apache and mod_wsgi. The database is SQLite. The operating system is FreeBSD, on a VPS hosted at Johncompanies.com. Comment-spam protection by Akismet. Vintage topo imagery from the Maptech archive. The markup engine is Markdown.

Pile o'Tags

ajax apple blogs browsers business design django ebay fp fun google hardware haskell intellectual property iphone javascript linux media mercurial mobile open source os x oscon oscon2007 palm php programming python rails rants reading risks ruby site spam textmate the well tips tools unix vcs web development web frameworks webservers wmassdevs

Stuff I Use

Akismet, del.icio.us, Django, dpaste.com, Emacs, FreeBSD, Freenode, jQuery, LaunchBar, MacPorts, Markdown, Mercurial, OS X, Postfix, Python, SQLite, Subversion, TextMate, Trac, Ubuntu Linux, wmii

Spam Report

At least 67529 pieces of comment spam killed since January 2008, mostly via Akismet.

Branching and merging in real life

At work I still mostly use Subversion for version control. Its main selling points: stable, performs as expected, integrates nicely with Trac, holds all our old stuff (legacy inertia).

Note that "pain-free branching and merging" is not on that list. (And don't give me the old "branching is cheap in svn!" line. It's not about the branching, it's about the merging.) A couple years ago I started also using Mercurial and plan to eventually replace svn with it entirely. The aspect of Mercurial that made my life better recently is its support for branching and merging.

The scenario: an important internal web app (in use all day every school day) needed some significant changes on a short timetable. Normally I'd work on the app thus: edit the staging copy, commit, update the live copy. I didn't want to take that approach here. I knew that during the development window there might arise unrelated urgent change requests; I wanted to keep the new code isolated during development, but also deploy and track those unrelated urgent changes. Branching seemed like the right approach.

I could have made a full clone of the app (hg clone mainrepo newrepo). However, handling environment dependencies (web server, PythonPath, database) would have added time and fussiness to the job, and time was in short supply. So, using Mercurial's named-branches feature, I made a new branch (hg branch newstuff) right inside the fully-functional staging copy of the app. That way I was able to develop and test as usual, secure that my unproven work-in-progress was not "polluting" the current app's revision history.

To handle "unrelated urgent changes" as mentioned above, I'd:

  1. Commit any current work on the "newstuff" branch
  2. Switch to the main branch (hg update -r default)
  3. Make the urgent change, test, commit, update the live copy
  4. Switch back to the new branch (hg update -r newstuff)

It took me a couple tries to understand how branch-switching worked, but it's simple: you really are updating your working directory to a new revision, it just happens to be a revision stored in a different branch from the current one.

It was fun looking at the graph (via HgWeb) and seeing my two parallel branches with their individual commits.

The moment of truth came at the end of the day Friday, when it was time to merge the tested and complete "newstuff" code with the current live codebase. It was dead simple, and effectively instantaneous. Condensed version: hg update -r default; hg merge -r newstuff; hg ci -m "merged new stuff". Followed by: update live copy and let out a big sigh.

Saturday, November 7th, 2009
+ +
7 comments

Summer Spam

Spam is occupying more than its customary share of my attention in recent weeks. I've long had a morbid fascination with sleazy human communication (hence Purportal.com). That makes the always-relentless stream of spam, though not exactly welcome, at least interesting.

Spam volume also seems to have increased during this period. The number of spam attempts my mail server rejects per day had been steady at around 3,000 for months. Now it's back up around 5,000 or 6,000.

I run my own mail server and fight spam via greylisting, blacklisting, and other strict technical rules. This setup rejects 99+% of the spam aimed at the domains I host, but some still gets through to me. Never enough to displace real mail, but enough to keep my little hobby-interest alive. Here are some of the spam highlights of my summer so far:

And finally, there was the phishing message I received today. It was a fake eBay notice, with the usual "click here to resolve the dispute" links. Those links were supposed to take the victim to a fake eBay page the scammers had set up (where the victim would type in all sorts of exploitable personal information). Looking at the message's raw source, I noticed something very odd -- the pages they were trying to link to were on an FTP server in Russia. Even weirder and better, the link code contained their FTP username and password! A minute later I was logged into their FTP server, looking at the one file there: the fake eBay page.

This was a darkly humorous reminder that the international spam-and-scam business is, from what I can see, a refuge for IT people (or wannabes) with poor skills and poorer ethics. So by this point I was kind of feeling bad for the incompetent underling who had put this thing together for his terrible boss.

However, I didn't let my compassion interfere with my sense of justice and fun. I replaced their fake eBay page with my own content, a much simpler message in plain text: "We are scammers."

Thursday, July 23rd, 2009

1 comment

SPF-enabled spam domains

Among the many anti-spam measures on my mail server -- which help me reject 5000 spam attempts per day -- is SPF. SPF allows domain name owners to specify which mail servers are allowed to send its mail. That makes it an excellent way to detect address forgeries, a favorite spammer tool.

One of the early questions raised about SPF was: won't spammers just buy their own domains and set up their own SPF records that say it's all OK? You can read the answer in the SPF FAQ, but the short version is: Yes, they will, but it won't give them a free pass.

That's because if spammers register a domain, publish SPF records for it, and send spam, they've identified that domain as one intended to be used for spam. Very good blacklist fodder.

With that in mind, here's a list of about 50 domain names that have recently been used to send me spam. All of these have published SPF records, and all the spam I received was from servers approved by those SPF records.

In other words, as far as I can tell, these are domains that exist primarily, if not purely, to send spam.

Update -- Here are the latest as of 2009-07-24: alg.com barrewardonline.com blueheavenbooks.com boudy.com eautocentral.com export2000.ro outpost.mm302.com qeentreeforlife.com ronaldvnash.com sistemas.com.ar smartserv.net solorpowernowme.com spig-int.com synergynetfour.com topproducerhelp.com truehouseinfo.com truelifeproducts.com unafraidrewardonline.com weathersearchontheweb.com

If for some reason a perfectly innocent non-spammy domain of yours has made it into this list, please let me know. (You might have to use my contact form, since I've already blacklisted all these domains!)

Wednesday, June 3rd, 2009
+ +
1 comment

Chess via iPod

I'm still loving my iPod touch. It's really a great little handheld computer. I'm able to do almost everything I need with the stock apps, but there are a couple free third-party apps that have earned a permanent place on it. One is the game Chess With Friends from NewToy.

Chess with God This is a version of what is also known as "postal" or "correspondence" chess. You make a move and send it to your opponent; your opponent makes a move and sends it back to you. (In this version, the CWF app rather than your mail carrier is the middleman.) You can pick somebody out of your address book, or ask the CWF app to find you a random opponent. Nice touches include in-game chat, step-by-step replays, and optional email or SMS notifications.

The human angle is what makes it fun. Most chess players have been periodically disheartened by computer opponents that beat humans (those who play at a mortal level like I do, anyway) coldly, soundly, and rapidly. The variety of human players that the CWF random-opponent feature delivers is a welcome change.

You get to pick your own screen name. People who know you can search by this name if they like, so it serves a useful purpose in addition to being a nametag. It also is occasionally the source of some amusement, as in the screen capture included here from the end of a recent game.

Monday, May 11th, 2009
+ + + +
2 comments

Aesthetics and computation

This evening, the Western Mass. Developers Group was treated to a talk by Ben Fry of Processing fame. It was excellent and inspiring. Having not much prior exposure to Processing or his work, I left hungry for more. (The title of this post is taken from the name of the group at the MIT Media Lab where Fry did his PhD work.)

I liked the graphical-REPL flavor of his live demos. Surprisingly, the feeling reminded me of being a kid flipping through Alan Kay's article about the Xerox Alto in Scientific American 30 years ago.

He gave a fun tour of creations by Processing users, with various highlights along the way including magazine cover art, a Superbowl ad, a scene from Minority Report, and the work by Robert Hodgin that was picked up by Apple for the iTunes 8 visualizer. Along the way he was concientious about giving his co-conspirator Casey Reas (not in attendance) his share of the credit.

Turnout was good, by our small-town standards: a full room, 25 people or so. Many had come out of the woodwork from local colleges (notably Smith and UMass). O'Reilly gave us a few copies of his book, which we had a drawing for at the end.

I found his work to be a heady mix of technical acuity, aesthetic commitment, and pragmatism. And I liked his dry sense of humor -- jokes that many non-technical audiences probably wouldn't have even known were jokes.

His work is especially interesting to me because I've straddled the design/enginering line most of my professional life.

At the end I asked him about this cross-disciplinary world of his, and whether he had observations about qualities that were good predictors of success. He thought for a moment. His answer, which included mention of a Harvard class he taught to a mix of art/literature/CS/etc. majors, began with one clear word: "Curiosity."

Tuesday, May 5th, 2009
+ + + +
2 comments