The Automator (This Old Pony #81)

March 12, 2019

Last week I shared some suggestions for reducing the bus factor or at least the risks posed by losing team members and their institutional knowledge, including:

Automate. Automate. Automate. The brilliance of automating, from tests to deployment, is that you don’t have to know all the steps or their sequence, just how to start it off. Now, at a base level you really ought to understand what’s going on, but just to get started you don’t need to. So if you have your tests and deployment automated and you have only one “automation engineer” it may be tricky to make changes if that person leaves. However you won’t be dead in the water in the interim because you don’t need a person to run the automated tasks, hence “automated”. If automation sounds daunting substitute “scripting” and you’ll get 80% of the benefit.

Emphasis added.

I got some questions about this, and it’d be worth exploring a bit more

what it means to automate your projects,

why should you automate your projects, and

how you should automate your projects.

What automation means

This sentence especially was confusing, although I’m glad it was included: If automation sounds daunting substitute “scripting” and you’ll get 80% of the benefit.

It presupposes a difference between automation and scripting which is at best a fuzzy distinction to make. But I made it, and out of a sense of stubbornness I’m going to stick with it. Automation is the bundling of process steps that can be executed together in a planned way. This perfectly describes scripts. But “complete automation” is when these processes can be run in response to events without direct human interaction.

Which is to say that if you haven’t any scripted support for your processes, then that should be your first step. But the next step is setting these up to run automatically in response to defined events, rather than simply by being kicked off from the command line.

Why bother automating

Let’s first posit that you have your own project, no one else is involved as a developer, a product manager, nothin’. Why bother setting up scripts to run tests, to deploy the code, to set up a working environment?

The first is that you’re likely to have to do these things again, and you want to ensure that you do them exactly the same way every time. You don’t want to forget a step, you don’t want to have to wait at the keyboard to kick off the next step.

You also don’t want to spend time or mental energy thinking about these things. You likely want a certain outcome and even knowing that doing X will get you Y, if it’s easy to skip X you likely will. Much like staying healthy, your best bet is making enforceable decisions ahead of time.

Automating is a way of not just ensuring better outcomes but spending fewer mental cycles on the work required to achieve those outcomes.

Now let’s posit that there are other people who either are or will be working on the project. Do you they need to hope they follow the same steps? Are you going to make them follow steps in documentation and copy and paste like code monkeys? Of course not! You can give them a single script that automates all the steps they need to follow to do the same thing you did.

This saves onboarding time, debugging time, and a lot of team stress.

And bonus: when something is automated it can often be “parallelized”, even if that means run sequentially while you do something else.

How do you automate?

Or, how do you start automating, and what do you start automating?

Well, there are a lot of things you can automate:

Tests

Other code quality control measures

Deployment

Code review

Configuration

Project management tasks
The way to start automating though is to focus on locally executed scripts, and by that I mean simple scripts you start from your own computer. There are two excellent candidates to start with: (1) test and code quality and (2) deployment.

Test and code quality is easier to start automating (more on this shortly) but the benefits to automating manual deployments are, uh, significant. Not only are manual deployments incredibly error prone, but automating them allows you to deploy in ways you couldn’t before, certainly not without a lot of tedious and hard to remember steps. Of course, if you’re deploying with Heroku or a similar platform-as-a-service, your deployment is at least mostly if not entirely automated out of the box.

Testing is easier to automate because I’m assuming you already have tests that you can with a test runner, whether that’s pytest, nose, or Django’s own test runner. This gets you most of the way there, and often all that’s lacking is a tie-in with your static analysis tools.

tox - or even make - can help here. A tox configuration file like below would allow you to run both your project’s automated tests and static analysis - in siloed, test-only virtual environments to boot, with the singular command “tox”.

P.s. I’m excited to dive into this area, and while I’m going to cover some of the pain points I’ve observed I can’t know what you’re specifically struggling with. Hit reply and ask away or just let me know what’s on your mind.