E-Scribe : a programmer’s blog

About Me

PBX I'm Paul Bissex. I build web applications using open source software, especially Django. Backstory: In the 1990s I did graphic design for newspapers and magazines. Then I wrote technology commentary and reviews for Wired, Salon.com, Chicago Tribune, and lots of little places you've never heard of. Then I taught photographers how to create good websites. I co-wrote a book (see below) along the way. Current story: I am helping turn a giant media corporation into a digital enterprise. Feel free to email me.

Book

I'm co-author of "Python Web Development with Django", an excellent guide to my favorite web framework. Published by Addison-Wesley, it is available from Amazon and your favorite technical bookstore as well.

Colophon

Built using Django, served with gunicorn and nginx. The database is SQLite. Hosted on a FreeBSD VPS at Johncompanies.com. Comment-spam protection by Akismet.

Pile o'Tags

Stuff I Use

bitbucket, Django, Emacs, FreeBSD, Git, jQuery, LaunchBar, Markdown, Mercurial, OS X, Python, Review Board, S3, SQLite, Sublime Text, Ubuntu Linux

Spam Report

At least 232284 pieces of comment spam killed since 2008, mostly via Akismet.

Switching from OS X to Ubuntu

In July, I switched from OS X to Ubuntu as my workday environment. For three years my personal MacBook Air had been pulling double duty, personal computer plus workstation at my job (each role with its respective user on the box). When the combined demands for disk space exceeded the 250GB SSD, I took that as a sign that it was time for a change. I work outside my office enough that an external HD wasn't a practical solution, and a USB key is too slow.

So, I requested a Windows laptop from the company stash. Step one: wipe Windows off it. Step two: install Ubuntu 14.04.

Here are some highlights and hopefully useful details about the switch and the new setup.

Losing LaunchBar

One of my all-time favorite OS X apps is LaunchBar. For people who haven't used it (or Quicksilver or Alfred, which are similar), it's hard to explain, but its slogan "Keep your hands on the keyboard" is as good a summary as any. Launch/switch apps, control apps, run web searches, manage clipboard history. It's like a fast command line for everything outside the Terminal.

As other users will tell you, once you're accustomed to it, trying to get work done without it feels like typing one-handed. It's possible, but you are used to thinking and moving a lot faster.

After trying Synapse, Launchy, and Gnome Do, I settled on Kupfer as the best replacement. This may be little surprise, as Kupfer is clearly modeled after Quicksilver, which is LaunchBar's closest competitor. However, having spent so much time with some others, I feel the the good fit is more than superficial. (As a bonus, it's written in Python, so hopefully soon I'll be hacking on it.)

After all that I still needed clipboard history. I found Diodon to be the best combination of useful, configurable, and minimal.

Not hating Unity

I know the new Ubuntu graphical shell has generated a fair amount of controversy, but in the end I'm finding it quite livable. A key piece of this is some of the customizations I detail here. However, for the most part the allegedly annoying aspects of Unity stay out of my face. I like the HUD a lot when I'm hunting for a particular command. My only gripes with the HUD are 1. I would like to be shown keyboard shortcuts along with the menu commands I'm finding, and 2. The default hotkey of Alt results in way too many unwanted activations (especially if you're an emacs user!). I solved #2 by remapping the key. Because you can do that.

Window management

I know if I were a truly elite Linux hacker I'd wield xmonad like a boss. I like xmonad a lot, but I'm not quite there for daily use.

On OS X one of my longtime gripes is how much you are tied to the mouse for window arranging. To alleviate this pain there I use a great app called Divvy to size windows and throw them around (including from one monitor to another).

On Ubuntu I've replaced it with a fairly simple trick: Compiz Grid keyboard shortcuts. For example, I set ctrl-alt-space for fullscreen, ctrl-alt-left for left half, ctrl-alt-1 through ctrl-alt-4 for the four quadrants. I also have a key that moves the front window to the other monitor. I set the keys in Compiz Config Settings Manager > Window Management > Grid. I'm pleasantly surprised how much mileage I've gotten out of this. (I believe I had to install at least one additional package to make this work, but I'm afraid I've forgotten what it was.)

Mail and calendar

Like many businesses, the one I work for revolves around a Microsoft Exchange server, so I had to be able talk to that. Not just email, but calendar too.

I tried Evolution for a while, since it was touted as an Outlook replacement, but I never managed to get it to talk to my calendar. I switched to Thunderbird and (with the help of a couple add-ons) that's working fine. Fine except for LDAP. As a stopgap for that I harvested addresses from my mail store using Email Picky4.

Other Apps

My main editor is the same as before: Sublime Text 2. One of the reasons I chose ST a couple years ago when I switched from TextMate was that it was cross-platform, and I was thinking about the possibility of this switch even then.

I do find myself using Emacs a fair amount though. I've always liked Emacs.

For storing passwords and other secure data, I love pass, "the standard unix password manager". I run it on my Macbook now too, and sync them via a private bitbucket repo (with separate branches for work/personal).

Music-playing is a motley combo: some Spotify (though the app's a bit flaky), some Google Play (I uploaded my whole iTunes library from my Mac), and Radio Tray for listening to my favorite stream.

For Lync chat (because some of my co-workers are Windows only), I use Pidgin with the SIPE plugin. Only real annoyance was having to manually assign "Alias" values to contacts so that they would be identifiable in the buddy list.

Unity "Dash" hotkeys are handy for quick switch between main apps like browser, IRC, mail, terminal, editor (e.g. I press Super-1 to switch to Chrome).

For backup, I'm using Deja Dup (via the built-in "Backups" utility). It's no Time Machine but it does the job.

Hardware

The machine is a Dell E7240 with 250GB SSD and 8GB RAM. It's a good solid package. It's quiet and fast. Battery life is modest at 2-3 hours, but the Air kind of spoiled me there.

To control brightness on my Apple Cinema Display, I installed ACDControl. I had to install it from source, and add a patch for my specific display model. Lookit me, I'm a l33t Linux hacker.

I haven't figured out how to run the machine clamshell mode (or if it's even possible). My workaround has been to raise my main monitor up on a couple fat books so that the laptop display can be open beneath it. Good enough.

Bottom line

I consider this a very successful switch. It's been about ten years since I spent my workdays in an entirely open source OS and desktop. My experience back then was good, but things are even better now. "The Year of the Linux Desktop" has become a running joke over the years, but for me it's this year.

Saturday, October 11th, 2014
+ +
2 comments

The standard unix password manager you never heard of

Recently I switched my work environment from OS X to Ubuntu (a post on that project is in the works).

For years I've been using the standard Apple Keychain app, which has several points in its favor: it's included with the OS, it integrates well with a lot of applications, and is not trying to "freemium" me into a paid tier. However, it's OS X only, which meant I had to find something new.

I wanted something that was cross-platform, simple, and capable of securely sharing the store across machines. Ideally I wanted it to be open source as well. I did not particularly need a phone app as part of the package.

There are scads of password managers out there, and I got many enthusiastic recommendations, but the best came via my co-worker Matt Simpson:

pass, the standard unix password manager.

The subtitle is somewhat aspirational, but not an unreasonable goal. It's a tool that does one thing well. It leverages existing, proven software like GPG and Git. Using it on the command line, you feel like it belongs. (Especially if you use shell completion.)

For shared storage, I set up a private Git repo on Bitbucket. I cut one branch for my work data, and another for personal. I installed it on my work (Linux) machine and my personal (OS X) machine both, and may well end up putting it on my FreeBSD VPS too.

Monday, September 8th, 2014
+ + + + +

The story of dpaste.com 2.0

Eight years ago, I launched a simple pastebin site written in Django.

In those early Django days I spent a lot of time in the #django IRC channel. I thought we should have a pastebin that knew how to correctly colorize our code, and which was written in our framework to boot. So I wrote one. Eventually its URL ended up in the channel topic, then in the Django source code itself.

Over the years, changes have been minimal. I switched from a Javascript-powered colorizer to Pygments. I added an API. I fixed things that broke. Mostly I just kept it running and usable. (I'll also note that many excellent new pastebins were created in those years as well.)

Of course I kept a growing list of "someday" improvements and fixes. Despite having no free time (full-time job, twin baby boys, 100-year-old house), a year or so ago I started hacking away on the items on that list. In very tiny steps.

Eventually I had enough in place for what I call the 2.0 release of dpaste.com. I presented a preview to my co-workers (many staunch dpaste users among them), and went live about three weeks ago.

I call out the fun new stuff on the about page:

One obvious change is the switch from auto-incrementing integer IDs (1.7 million at last count) to 7-digit base-32 IDs like "1S2BP7E". This ID is generated by hashing some of the paste data fields, using MurmurHash. I wrote a simple library called basewhat to handle the base-32 conversions.

Some other improvements are less visible but still important:

Every good developer knows that version control plus a good test suite help you go faster with more confidence. If you also deal with the operations side (i.e. deployment), I'd add Fabric to that list. By reducing error and tedium in maintenance and deployment tasks, Fabric likewise increases your likelihood of doing them consistently and correctly. If pulling the latest code to the live server and restarting the app is a single command I can run from anywhere, I'm going to do it more.

Thanks to all the authors of the open source software that this project is built on. And thanks to all the users, who motivate me to keep it going just by using it.

As they say in customer service land, "We know you have a choice of pastebins, and we thank you for choosing dpaste.com."

Friday, May 16th, 2014
+ + + +
7 comments

Creativity and Constraint

O'Reilly Media recently asked in their "Programming Today" newsletter:

"For many old-school software engineers, developing code has always been as much an art as a science. But as the industry focuses more on practices such as Test-Driven Development, and Patterns become the lingua franca of programming, is there any room left for real creativity in coding? Or, has it become an exercise in cookie-cutter production, putting together components in new ways, but without any real room for individual style? Share your thoughts with us..."

My response:

The idea that TDD or Patterns are in opposition to creativity in coding is a false dichotomy.

Our work is inherently about working effectively within constraints.

Those constraints may be related to the business need -- time to delivery, performance requirements, budget limits that circumscribe personnel or technology choices.

Or those constraints may be inherent in the technologies our projects use -- we may have bigger or smaller standard libraries, larger or smaller variety of developer tools, greater or lesser impediments to integration. And we have varying levels of access to the source of software products/services/components that our projects are built on.

I'd say these facts are no more an indication of impinged creativity than are the material facts of other craftspeople. A carpenter must work with the limits of wood as a material -- but a good carpenter knows and leverages its strong suits as well.

If a software shop is managed such that TDD or Patterns becomes a doctrine that must be followed without regard for its effectiveness, that's a management problem with that shop -- not an indictment of the tools and techniques being abused.

Great art and craft and science work of all sorts always has constraints. Creativity manifests itself in response to those constraints.

Those are my thoughts.

Friday, April 26th, 2013

Booktools

A little-known bit of trivia about our book, Python Web Development with Django: we wrote the manuscript in Markdown.

I think it was my idea. One of the major motivations for using a text-based format -- versus the unfortunate de facto standard, Microsoft Word -- was integration with good developer tools and workflow.

Our manuscript and all our project code was in a Subversion repo, so each author always had the latest updates. HTML generated from the Markdown files was great for generating nice printed/printable output too.

We could have used any number of similar formats: Markdown, Textile, reStructuredText. If we did it again we'd probably use reST plus Sphinx. That would grant all the same advantages, plus give us a little more formatting flexibility and tool support.

This workflow enabled certain kinds of programmatic action on our text, notably two things: automated testing of the interactive examples within the text, and automated extraction of example snippets from source code files.

I wrote scripts for each of these tasks. I've cleaned them up a little, to try to make them a little more general-purpose, and published them (with minimal example files) in a bitbucket repo: http://bitbucket.org/pbx/booktools

There's a little documentation in the docstrings of the scripts. Here's the summary:

To test code snippets in the manuscript file: test_snippets.py example/text.txt

To extract code from source files into the manuscript file: try_excerpt.py example/text.txt

Authors, make use of this if you can -- or maybe even better, take inspiration from the idea and implement a system of your own.

Tuesday, September 14th, 2010
+ +
2 comments