Month: September 2007

I spent the whole day today stepping through DomainKeys, DKIM, and SPF for a customer to make sure that they were all set up and working properly. I found a couple of the existing tools available on the Internet didn’t test them properly and didn’t give enough explanation when they failed.

So, I went about creating some of my own tests using a bunch of Perl modules. I finally fixed a couple issues that I think were causing problems for this customer. Gmail and Yahoo are now delivering the messages anyway. Hotmail is still a mystery (no surprise since its from Microsoft). Their troubleshooting website is full of talk about how great their service is, but there is a huge lack of anything technical on the site.

Anyway, I made all of these nifty testers and figured that others might like to use them. So I created a way for others to send mail to it, and a web interface that you can view all of them results. I’ve added it as a link to the top of my website.

I was trying to verify a DomainKeys message from gmail with the Perl Mail::DomainKeys module and it wouldn’t verify for some reason. After digging into the module a bit, I found that gmail doesn’t send the ‘q=’ parameter to specify the method to look up the public key. Obviously this should default to ‘dns’ since it is the only currently supported method.

I modified Signature.pm and added this to line 56:

$self->{‘PROT’} = $self->{‘PROT’} || ‘dns’;

And now the messages are verified properly. I’ve emailed the module’s author about it, but am not sure if he’ll bother with releasing a fix. DomainKeys is being dropped in favor of DKIM.

I spent a good part of the day today working to make sure that a DomainKeys and DKIM server was properly signing messages. Here’s some of the confusing stuff I sorted out:

First off DomainKeys and DKIM are different slightly different technologies that do the same thing. My understanding is that Yahoo, Gmail, and others came up with DomainKeys as a solution to to solve email forgery. The EITF then tweaked the original DomainKeys spec a little and made their version called DKIM (DomainKeys Identified Mail). The two are very similar and I haven’t found anything that describes their technical differences.

First, I guess it helps to know basically how DomainKeys works. Essentially the administrator creates a Public/Private key pair on the mail server. The private key is used to sign the message while the public key is posted as a DNS TXT record for receiving mail servers to use for verification. Receiving mail servers then use the public key from DNS to decrypt the signature and make sure that the signed headers and message body were not modified in transit. I’ll describe DomainKeys here, and just understand that DKIM might implement things slightly differently.

Note that this doesn’t actually do any spam checking on its own. It simply verifies that the message came from an who it claims and wasn’t tampered with along the way. It is intended to be used for fighting phishing, and creating a reliable reputation-based services.

Originating mail server:

When a message is submitted to a mail server for signing, it uses the ‘From’, and ‘Sender’ headers, as well as the message body, in combination with the Private Key to generate a DomainKeys signature. This signature is added to the rest of the email headers and looks something like this:

You’ll notice that the header contains several name=value pairs that describe things about the signature. Here are some important options that may be used in the Signature:

a= describes the hash algorithm used to create the key (usually rsa-sha1)
s= contains the selector (described later).
d= the domain to use for signing
q= the method for retrieving the public key (only dns is currently supported)
p= the base64 encoded signature
See section 3.3 in the DomainKeys specs for more details or Section 3.5 in the DKIM spec

After the MTA adds these headers, it forwards the message as normal to the recipient.

The receiving mail server can then perform some validation on messages that it receives which contain a DomainKey-Signature header. To do that the receiving MTA receives the message.

For every message (regardless of weather or not it has a DomainKeys-Signature header) it receives, it looks up the DomainKeys policy record for the senders domain. This is a text record with the name ‘_domainkey.DOMAINNAME.TLD’ where the ‘_domainkey’ part of that is always ‘_domainkey’. If the message does have a DomainKeys-Signature header, the -d parameter should match the domain portion of the ‘From’ header.

The policy record for a domain has these options:Section 3.6.2 in the DomainKeys spec and 7.4 in DKIM specs
Of interest include:
o= can be either a hyphen (-) to indicate that the domain signs all email (and non-signed messages can be treated as invalid) or a tilde (~) to indicate that the domain signs some messages.
t= Set this to ‘y’ to indicate that you are testing. Receiving mail servers are not supposed to penalize you if the message does not verify and the ‘t’ flag is on.

Of interest here are these options:
k= The key type (always ‘rsa’ for now)
g= (dkim only) used to limit the addresses that a public key can be used for. For instance to outsource some email without giving the third party permission to sign any address.
n= human-readable notes (not to be used for anything)

I’m not sure how I stumbled on it, but I came across this really old version of the BackRub home page – which is what evolved into Google. Its kind of interesting to see that some of the links that work and see some of the things that they mention.

In playing with some routing last night, I needed to filter the default route that we received from the BGP announcements from our peers. I didn’t find exactly what I was looking for via Google, so I began experimenting. After a dozen or so unsuccessful changes, I came up with this that actually accomplishes what I was trying to do:

I have something like 30 domain names that I own, so I’m always looking for a way to save a few bucks on registrations. GoDaddy.com always has some special offer for a percentage off, or reduced prices. I used to keep all of their emails so that I could go back through them when I needed to buy something. I just came across this site that displays all of the currently active and recently expired coupons. Now I can unsubscribe to their newsletter and keep it from filling up my inbox.

Also, I stumbled on the ‘bulk pricing‘ on godaddy’s site. When renewing 6+ domain names at a time, you can get a discount as well. Over 20 domains and the prices are about the same as with discount codes.

About a week ago, I subscribed to Verizon’s EVDO service so that I can get online from practically anywhere. The first few days, I had it running on a Windows machine and that was pretty simple to install. I finally got around to putting Ubuntu on that machine and getting it working under Linux which was also pretty painless.

I specifically chose the 5750 because I found plenty of online documentation for getting it working. Specifically this post, which has some pretty simple instructions. Within about 30 minutes, I had configured my fresh Ubuntu install to work fine with the card. It’s not perfect yet and at this point I still need to run a couple manual commands to get it to connect each time. Also, I’m not sure how to make it cleanly disconnect so that I can reconnect to a wifi service when that is available. I haven’t played with it much past getting it working though, so I’m sure I’ll figure out the rest soon enough.

Performance seems pretty impressive. Speed tests have usually been around 500-700 kbps down and 120-200k up. I’ve seen it as high as 1.6 Mbps down/700k up though. When sitting stationary, latency has been around 100ms to google.com, compared to about 40ms on my Comcast connection. That is decent enough to work with an interactive shell on and to use vim remotely without too much complaint.

I just finished testing latency during a 40 minute car ride. I pinged google the entire way home for a quick test. Although not as impressive overall, I was still impressed that it stayed connected and had less than 1% loss. Latency got as high as 6600 ms though, and the average latency was 272 ms, so that would be more difficult to do something interactive like a remote shell.

Overall, I’m impressed so far. Verizon has their 30 day test, so that has been nice that I had the chance to test everything out and would be able to cancel if necessary. At this point though, I’m satisified with the ease of getting it working under Ubuntu, and the performance so it looks like I’ll be keeping it.

Essentially, each filter has a ‘new’, ‘cur’, and ‘tmp’ directory. tmp is used when writing the message, and then the file is immediately moved to the ‘new’ directory, and the S=xxxx part added with the file size in bytes. Once a file is read by a mail client, it is moved to the ‘cur’ directory, a ‘:2,<FLAGS>’ parameter is added, where the flags can be used to mark the message as read, replied, deleted, etc

In the maildirsize file, the first line contains the quota size in the format xxxS,yyyC where xxx is the total size in bytes, and yyy is the number of messages. So a quota of 1048576S,1000C would be either 1 MB or 1000 messages (whichever occurs first). Then, each line after that contains two numbers. The first is a size in bytes, and the second is the number of messages.

Each time a new message is saved, a new line is added to maildirsize with its size. The total quota is calculated by totaling up the two columns. Occasionally, the maildirsize file is recalculated from scratch.

I’m about ready to wipe out my laptop’s hard drive and reinstall, but wanted to back it up first, just in case there is something I need on it in the future. I was searching for open-source alternatives to Symantec Ghost and came across CloneZilla which looked like it would do the trick. It is available as a Linux-based Live CD so that you can just boot off it and go. With just a few minutes of playing with it, I was able to back up my entire laptop hard drive to a Samba share on a Windows PC.