Processing MD5 Suppression Lists - a Tool for Affiliate Marketers

Affiliate marketers will from time to time have to process what’s called an “MD5 suppression list”. In brief, an MD5 suppression list is a list of email addresses which a marketer must remove from her mailing lists, in order to comply with the CAN SPAM Act of 2003, and respect the rights of individuals to opt-out of email marketing campaigns.

An MD5 suppression list is simply a file containing a long list of MD5 hashes of unsubscribers' email addresses, the hashing being a security measure designed to prevent unscrupulous marketers from using suppression lists themselves as sources for obtaining more email addresses to use in email marketing campaigns.

To use a suppression list, an email marketer must compare each hash in the suppression list against an MD5 hash of each contact in her mailing lists. A matched pair of MD5 hashes indicates that an email address has been found in the suppression list, and thus must be removed from the marketer’s email lists. (The mechanic here, obviously, is similar to how user passwords are hashed before being stored in a database.)

Recently, at work, I had to process a 2 gigabyte suppression list (of about 62 million rows) from Groupon. To my surprise, I didn’t find any readily available tools to do this, and thus, rolled my own.

Deploying Wordpress: Syncing Files in a Multi-Server Installation

Recently, some of my company’s WordPress sites have become so popular that I chose to migrate them onto a multiple-webserver deployment system in order to keep up with the traffic.  I encountered some interesting challenges while setting this up, so I figured I’d document them here.

Making Wordpress Domain-agnostic

I like Wordpress a lot. It’s one of my favorite open-source projects, and I use it often for both my professional and personal projects. It’s been my go-to web development framework for a number of years now.

There’s one thing I don’t like about Wordpress, though: the domain to which a Wordpress site is deployed is saved as a setting in its database. I don’t think that was a good design decision, because it makes it painful to move a Wordpress site from one domain to another. This shortcoming is especially evident if you’re trying to develop a Wordpress site on one domain, but would like to deploy to another. (For example, I always set up my local sandbox such that the WIP lives at example.dev, while deployments are pushed to example.com). I really wish Wordpress had been designed to path against its own document root, much like MediaWiki (another great piece of web software).

A while ago, though, I came up with a little hack to make Wordpress do exactly that.