Dan Nordquist on Technology, Work, and Music

Main menu

I’ve been giving a talk around town about DevOps and what it means for me as a software developer. In short, I’m really excited about the idea that the wall between “dev” and “ops” is softening in a real way.

At a recent event, someone raised their hand to ask how an organization can know if they’re “doing DevOps correctly”. Is there one pattern or practice that separates the people who “get it” from the novices? I was speechless! Flatfooted. I wish I’d had time to whip up a blog entry on the spot.

First of all, this is obviously more complicated than that. It makes me think of “one weird trick to lose weight” or “the three things that every successful person does before 7am”. Easy answers might point you in the right direction, but they’re no substitute for a complete understanding.

But: if I had to answer the question, I think it hinges around maturity and automation. If you are making progress towards automating what you can, and manually intervening when you have to, then I think you can claim you’re going in the right direction. If you don’t have time to do the things that will save you time down the road, then I don’t know that you can expect to develop maturity in the DevOps space.

And, of course, “doing it right” is no reason to let up off the gas and rest. Continuous improvement means continuous experimentation and change.

Google Mail is great. It’s probably the best combined mail-service-and-mail-app in the world. As soon as I think someone’s coming out with something that’s maybe-just-as-good, GMail kicks it up with something inimitable.

And, of course, their inimitable thing might be integration with a product that Google owns. After not-long, I was using my Google credentials to sign into apps. Then you had to sign into YouTube with your Google account. Then I was getting pop-ups and emails about Google+, a social network on which I’d never really found any of my friends.

However, as they say, if you aren’t paying for a product, then you are the product. Google’s goofy history of increasingly arbitrary decisions (decisions that never gave benefit to users, beyond giving Google more leverage with advertisers and thus paying for more years of GMail) finally got to me and I had to investigate alternatives.

Here’s what I learned:

You’re replacing your mail service and your mail app. I really like Google’s mail app, and I don’t really know all that much about mail services, so I knew I needed one that could ably replace both pieces of it. (And not necessarily the mobile apps and chat service and calendars and stuff, but we’ll get to that.) Your favorite feature of Google Mail (video chat, stars, all those apps that are built on top of GMail now) probably simply doesn’t exist elsewhere, and you have to suck it up and let go. (That’s how they get you.)

That sort of thing starts at a couple bucks a month. I didn’t really want to spend $10/month on email, when I already spend money on web hosting and owning my own domains. Fastmail has a great rep and fits the bill – because I wanted to use my own custom domain, I had to select one of their higher tiers, but

There’s a whole world of mail filtering I had no idea existed. I said that Google’s strengths aren’t matched elsewhere, but I’m having good luck with Fastmail’s spam filtering. But: do you set up mail rules to filter social, commercial, and pestering emails out of your inbox? One of Google’s inimitable features is that they will handle this for you. However, Fastmail supports something called Sieve which actually runs all mail through a script to determine where it should go. Look at this syntax:

It’s not for amateurs (so there’s a GUI where a lot of this can be managed) but, for me, this is amazing. I keep a list of known noisemakers, commerce hounds, and social domains that makes filtering a breeze. (Actually, it’s not a breeze: maintaining the script and its data is a little obnoxious. But it’s mine, and it’s not written by Google, and it can’t be corrupted simply because advertisers have paid enough to defeat it.) I’m down to really truly inboxy stuff in my inbox, which is a huge deal.

It is a lot of work to change your email address. If you’re going to forward your old email to your new account, that’ll work, but I really wanted to be able to sunset my old email accounts and move forward with something that was new. (I’ve had my Google email address for 10 years, and my other main address for 16 years.) So I bit off everything at the same time: new address, new service, new app.

I watched my old addresses (which I still forwarded, naturally) and, one-by-one, went about the business of changing the email addresses by which they new me. From hardest to easiest:

People. If you want to email a friend, how do you remember their email address? You might have a contact manager or you might just find an old email from them and reply to it. (Google Mail will suggest commonly used email addresses if you just start typing someone’s name.) None of these are updated automatically when I send out my “hey I got a new address” email. People are the worst.

Companies who don’t give a crap. Lots of companies send email with no footer letting you get back to their site and edit your profile. All I could do was unsubscribe, and so I did. (Potential scope creep in any project: if the app should send an email, are you ready to think about every aspect of sending email, including allowing the editing of an email address?)

Companies whose systems are tightly coupled to your email address. I have a number of accounts at places where editing one’s email never occurred to them. Apple was particularly bad. I’m still (eight months later) trying to convince iCloud who I am across all my devices. (How’d they not see that coming?)

Companies who get it. I was actually pretty impressed by a lot of organizations who let me change my email no problem, so long as I clicked a link in an email to the new address, and almost all of them sent an email to my old email address just in case. Better yet were companies that had a concept of “membership” that would allow them to keep an array of email addresses. For a company like LinkedIn, that’s key: they want you findable by your “work email” no matter how many companies you work at over the years. But Microsoft has a similar system, and it’s really nice to be able to add a new email address without being logged out on all your other devices.

Long term, I’m really happy I did this. It’s not without effort or expense, but it’s where I am right now: too concerned with being sold to advertisers to sit comfortably with an ad-based service, and not wanting to run a mail server in my house.

I think it’s the case that a lot of developers spend as much time in Outlook as they do in Visual Studio (or their IDE of choice). As a matter of fact, I’m fairly sure that there are more Java developers stuck using Outlook than there are MS developers who are on a different business email platform. (Because what’s the other business email platform?)

I’m writing this up because I just found a bug (in my workflow, if not in Outlook), and it had certainly bitten me a few times before. But, before I share what I found, I’d like to share a couple of Outlook observations.

It looks like Outlook’s gone away in Windows 8, but that’s not true at all. People open up their Windows 8 machines and fire up all of the full-screen, touch-friendly, formerly-known-as-Metro Mail client, and find that they can’t really do as much as they could in Outlook. Well, Mail is free like Solitaire was, and Outlook costs $100. So there’s your answer. I think they still expect you to install Outlook if you need it, and you do.

Outlook plug-ins are to be avoided. I don’t know if it’s Outlook’s API/SDK, or if QA is harder with integrated software like Outlook plug-ins, but I’ve had really bad luck with them, even if they come from Microsoft. (One exception would be the Evernote plugin, which does exactly one thing – save what’s on the screen to Evernote.)

Folders are for the weak. If I see you fishing through a mess of folders (per project, per sender, per month) it makes me sad. Old emails are empty husks or shells: they contain information, and your job is (probably) to get the information out of there and put it somewhere useful. (Not a folder.) I did a lot of this, too, until Google Mail trained me to search instead of sift.

Make only as many changes to Outlook as you will remember if you wipe it out and start again. It is frustrating and confusing that Outlook does not remember your email signature (this is an application setting) when you switch from one computer to another. Properly configured, it remembers every email you’ve sent and received for a decade, but if you deck out the customizable ribbons (y’know, like they made them customizable) that’s something you have to back up and restore yourself. I generally live pretty close to software defaults anyway, but this is one area where I’ve had my heart broken enough and it’s not worth it.

Archive, don’t delete. Speaking of Google Mail and customizations, I also got trained to get everything out of my Inbox and into an Archive folder. This is one of the rare modifications I’ll do to a new Outlook setup. True garbage goes into Trash, but everything else should be in the Archive for searching. (Because sometimes you need to refer back to the empty husks of email to refresh your memory.)

One weird email trick to save your career. My other must-have modification is to set a one-minute delay on every email I send out. 60 seconds to reconsider that snap answer, reword an argument, or recant the response to a meeting invite can make a huge difference. Follow these mail delivery delay instructions, probably, and you’ll thank me at least four times in the next year.

So what’s the hiccup? My problem has been with the last item: delayed delivery. Outlook implements this (client-side, frustratingly) by setting a “sent” time that’s one minute in the future. So it sits there in the Outbox, waiting for its sent time to arrive, and you can “unsend” it by simply opening it. When it’s opened, that unsets the “send” flag. (When you send it again, it sends without delay, so don’t count on getting another 60 seconds to change your mind.) And I was noticing that more than a few of the emails I would send would be stuck in the Outbox, not leaving. This has been going on for years.

And what’d I just figure out? It happens when you combine wicked search skillz with that delayed delivery. A search for an old email (about “teapots” let’s say) will pull up a list of everything in Outlook’s influence that contains that word, so Inbox items, Archive items, and even Drafts and Outbox items (and, arguably, it shouldn’t). And, like I said, opening or viewing an Outbox item unsets its “send” flag. What was happening was that the search window’s live preview was “opening” my just-sent email, ostensibly assuming I might edit it, I guess? And it didn’t happen all the time because I generally don’t send emails that match the search I have open in the background, but it happened often enough that I knew it was a problem.

I don’t have an answer for that problem (I’ll probably just stop leaving search windows open for so long), but I did get a blog post out of the deal.

Saw this over at Daring Fireball. Disappointing, because I love John Gruber’s writing. I’m a two-spacer, probably because I was taught that in school, but also because one-space can be confusing or ambiguous with abbreviations.

“They sell candy in the U.S. Mint.”

I will not repeat the arguments for either side here, but the one-spacers have yet to convince me that two-spaces is not a useful and clear standard. They lecture the Internet at large about history and misinformation and authority and foolishness (and stubbornness! as if they themselves are not also stubborn!), but the mint-candy phrase above is inarguably ambiguous.

(And, of course, whatever I write here, it gets turned into HTML, which has to be tricked into displaying two-space formatting. It’s out of my control, but I can live with it either way. When I am in control, I prefer to make the reader’s job easier.)

Some mail services allow a user to append a tag to their email address (e.g., where joeuser@example.com is the main address, which would also accept mail for joeuser+work@example.com or joeuser-family@example.com). The text of tag may be used to apply filtering and to create single-use addresses.

I see developers get this one wrong all the time. And, actually, they don’t get it wrong: they copy someone else’s regex or validation method, and never bother to test it. If I’m filling out a form online, I’m probably going to give you my Gmail address, and I’m probably going to inject “+yoursite” before the “@” symbol. That helps me track who’s abusing my trust.

But: you have a requirement to “reject” invalid email addresses. What’s a valid email address? Well, to get completely pedantic, lots of things are technically valid email addresses that you’ll never run into. This is one of those cases where I’d actually rather see a set of tests — good emails and bad emails — rather than a “requirement” (“email should be valid”) or the specification itself. (Are you doing anything useful with that email address? Are you confirming that it belongs to the user? Because if you can’t answer those questions, a vague requirement about “valid” emails is just a distraction.)

Truly, the requirement should be based on the audience and expected input. Are you creating a site where people might be typing in their username rather than their email address? (Do your users confuse their mailing address with their email address?) Protect yourself against the scenarios you’re likely to run into, and then pour your energy into other ways to make your form better. (Not writing the fix for the edge case that means an.”unusual\ example”@clownpenis.fart doesn’t pass, because I guess it’s a valid email address.)

A friend wrote to ask “what are ten questions you ask at the start of a project?”

My jobs, over the course of the past few years, have let me work more directly with people at the start of the process, so this is a question that I’ve thought about a lot. (Over the course of the years, I’ve started and refined a document that eventually numbered 48 questions, so I’ll have to make my questions here count.)

What Do You Know?

First off, no set of ten questions is going to work for everybody, all the time. I’ll focus here on custom software development on the web, but even that’s different than it was five years ago. Today, you’ll want to ask questions about how the development effort integrates with mobile strategy and any social media presence. (Both of these things would have been intriguing and futuristic until somewhat recently.)

The Questions, Already

Where are you at in your process? It might be that the client has the website totally mapped out, wire-frames wire-framed and servers procured. It might be that the customer just has an idea, and needs help with strategy, design, content, and analytics. If you ask, you’ll get an answer, even if the response is “huh?”

What decisions have you made about technology? Sometimes a project is really complicated because of a technical decision or legacy system that was made before you got there. Maybe it’s gotta be done in Plone (you know, Plone, the CMS that’s built on Zope).

What’s going to happen after the site launches? What is the customer going to do with their new / relaunched site? Are they expecting to see orders roll in? Are they going to want to change content? Are they planning on a phase 2 after phase 1 is done? That changes not just the complexity of the development scope, but the planning, too.

Roughly, how many pages and templates are we talking about? This is the start of nuts-and-bolts web stuff. (Swap this out if you’re building something else.) This is really just for a sense of scale. Sometimes you’ll hear “I can’t tell you right now, I don’t know” and what they mean is “it’s either 7 or 8, depending on what Gary decides”.

How should this behave on mobile browsers? The answer to this might really surprise you. They may have decided that they can forget about old phones, or it might be that the CEO has a BlackBerry, or it might be that they’ve never really thought of that. On the other hand, they might say “yes, pixel-perfect, across every browser”, and at least you know what you’re dealing with.

Are there internationalization, accessibility, or regulatory challenges we should be aware of? You want to expose multi-lingual stuff immediately. Otherwise, you just want to know if you’ll be working in a space with a special-needs audience or government oversight. This can triple an estimate of effort.

What kinds of interactions, customization, or membership do we need to think about? If you’re talking websites, it’s pretty clear that there should be N pages, utilizing M templates, and some kind of navigation that ties them together. But does the site support search? Contact Us? Some kind of calculator where you determine what your mortgage insurance is going to cost? A marketplace where people shop for insurance? You’ll want to know.

What other systems does this site connect to? Maybe the most important question. Generally, I hear “none, no other system” and then I ask about Google Analytics. That’s a system, and the clients will be interacting with it (unbeknownst to them). And email — does this system kick off emails at any time for any reason? That’s another system.

Who’s going to do what, now and after the development? Maybe this isn’t relevant everywhere, but sometimes you’re selling development. Sometimes you’re selling a team, and sometimes you’re selling an entire project. So, do they have PM and QA representation they’re bringing? Will you be working with an onsite IT team to deploy, or will there be a fierce rivalry between your team and theirs?

When are we starting, finishing, and getting paid? I sometimes leave this for other people to ask and sort out, but these details have to be asked at some point. A word of warning, here: they’ll always say “we don’t know what this should cost, you tell us”. No real project doesn’t have a timeline / budget. (Or: if you have to ask, you can’t afford it. In my experience, this is unfortunately always true.)

Moving Beyond the List

The script above evolved over the course of years. Some questions worked, other questions turned into others, and some questions never got me where I wanted to go (so I don’t include them above). If you don’t write this stuff down, you minimize your chances of evolving and improving your process. (It’ll evolve, I guess: randomly. Wildly. Aimlessly.) Give your list of questions some careful consideration after each interaction, and you’ll be naturally drawn towards making it better.

So there’s been, y’know, talk about JavaScript at work. With JavaScript encroaching on the server-side more and more, I’ve seen chatter about unsuitable it is for server systems.

I was one of those guys. I didn’t care if I could do it client-side. Give me a post-back and a code-behind and I’ll get it done. Or: hey, nice client-side implementation of that. You know the next Gecko or WebKit or IE will mangle it, right? Hope they don’t release a new version tonight.

My perspective today is that JavaScript is the language we’ve got. If you hate it, well, there are lots of jobs working on systems and applications that don’t touch web browsers. However, if web browsers are where your audience is, then you’d better get comfortable. I’d bet that most of your beef with JavaScript (if you have beef) is actually with browsers. They’ve been, well, inconsistent. But JavaScript is good (as in “flexible” and “expressive”), and getting better. (And I don’t see a real competitor for it.)

In terms of that encroachment, I think stacks like Node are really intriguing. I think more traditional stacks will definitely catch up, but I think it says a lot that the first group to patch together something like that (a server that was light, event-driven, and I/O intensive) used JavaScript (and V8) to do it.