The Era of Automation and the Lone Orchestrator Has Begun

How to run websites, for mortals who value their time and sanity

Let me tell you how I did this website thing in the old days—that’s the mid-1990s, kids.

I cooked a bunch of HTML files, with plenty of hard-coded URLs sprinkled in, and seasoned the mix with a bloated style sheet; launched an FTP client, preferably one with buttons to click on; and then uploaded my beautiful goop to a directory on a web server—that was all I needed and could afford.

Then the web hosting company performed some black magic rituals and the universe—OK, just me—marveled at my creation.

You can still play that game, that is, if your name is Timmy and the website belongs to your cat, Lassie, but if you intend to run a website where other adults do, you know, important stuff, as loyal Tank would say: “Timmy, that is loco.”

Everybody loves to fill their days with error messages, right?

These days, real-world web applications must be resilient and perform well; may involve multiple web servers, efficient caching mechanisms, and a host of other services; and have to query classic relational databases, NoSQL storage, or both.

For more involved configurations, you’ll need tasks queues, extra development libraries—no, not that version, the other one—and access to a myriad of software components and web services.

Oh, and did I mention that everything has to keep running if any part breaks? I’m sorry, Timmy, there’s no room for you and your cat in this, but we’ll keep you in our thoughts (no, we won’t).

If you are down with setting up an architecture like this by hand, expect to burn countless hours typing tedious, prone-to-error commands and watching messages scrolling down all night, like a disappointed Cypher. I guarantee you plenty of silly mistakes and constant suffering. What? You like it that way? Stop reading and go back to your basement.

Waits for masochist visitor to close the browser.

Still here? Cheer up, there’s a better way to spend your crammed days: You’ve got to automate and orchestrate.

Don’t repeat yourself, I repeat: don’t repeat yourself

Automation means describing a task, usually a single one, to run without human intervention. Any task that makes you repeat the same steps every time is a candidate for automation: launching a web server, refreshing configuration files, adding a back end to a load balancer.

Orchestration, on the other hand, refers to the arranging and coordination of all the automated tasks. It’s a workflow, such as running a test suite after a commit, deploying the latest code to production, or syncing data with staging. In human speak: Orchestration is the conductor waving the baton to get the best out of the musicians.

There are many tools to help you with this, and the ones I like are mostly open source. I used to set up my local development environments with Vagrant and its provisioning system but then I was lured to the containers world by Docker and now I see whales everywhere.

Pay attention to the tasks that you find yourself repeating every day, especially those where you always make mistakes as they are the perfect opportunities to introduce automation and orchestration. Yes, you will spend extra time at the beginning but will save effort and headaches in the future.

Ideally, you shouldn’t communicate directly with each server or service to perform repetitive tasks, and just one command or—even better—a triggered event should suffice. If you get there, you may want to take the extra step to continuous integration but that’s a story for another day.

To infinity and beyond?

You can go as far as you want building your architecture but don’t get carried away trying to make it perfect; remember you still have a website to run. I like to automate and orchestrate just enough to launch, deploy code, and achieve high availability, and then I continue working on other parts of the project knowing that I can return later to teach new tricks to little Skynet.

Don’t fret about leaving a few things to the experts, such as scaling Elasticsearch or maintaining a Redis cluster; instead, focus your time and energy where they are the most effective—let’s say, writing software or content. This is critical if you are running a one-person operation.

In sum, you are in charge: make your software work for you.

What’s that, Timmy, you just started writing your own Ansible playbooks? Beautiful. Now get rid of that silly cat and join us in Zion.

Related Articles

There’s a scene in Kill Bill: Volume 2 in which Budd’s boss says to a young, lady employee, “take a hit, be somebody, baby.” I love that scene.

Many a doubtful souls among us may find the idea of being somebody, whatever that means, enticing, so I’d like to start with the obvious public service announcement: a line of cocaine isn’t the way to go. That’s just Tarantino messing with the weaklings.

So which way then? Why, of course, be the best at whatever you do, right? That’s a fair assumption—be the smartest and be the fastest, because only by being perfect and creating perfection you’ll guarantee success. Right again, aren’t I? (And I expect you to be high-fiving the screen by now.)

The end? See you in the next article?

Hold it there.

I have my doubts about this perfection business; in fact, I…admire imperfection.

I’m not going to get into the topic of success today—I’ll get back to it in future pieces—but I’d like to talk about the relationship between perfection and the road to expertise. And to help me illustrate my point, ladies and gentlemen, the Beatles.

On Monday, July 10, 2017, New York City woke up to what Governor Andrew Cuomo had described as “summer of hell.” An eight-week period of train track repairs at Penn Station in Manhattan, expected to cause reduced service across the Tri-State Area.

I couldn’t think of a worse way to start the season or a better excuse to discuss remote work so I turned to the New York Times to have a better gauge of the situation.

Let’s see.

I'm Hooked, Count me in

Subscribe to Zinibu Weekly and keep up to date with my latest articles and advice for digital publishers and Internet businesses.

Needless to say, I'll keep you email address safe and will never share it or use it to spam you.