In this episode, we baked the Celery worker and beat scheduler tool into the Shiv app. This is one more step on the path to simplifying the set of tools on the production server.

I started the stream by reviewing the refactoring that I did to conductor/main.py. The main file is used to dispatch to different tools with the Shiv bundle.

The refactored version can pass control to Gunicorn, the Django management tools, or Celery.

To make Celery work, we hooked worker_main to the Celery dispatch function.

Once worker_main was set, I generated a new local Shiv app to test things out. Since testing could require building the app multiple times, we took a detour to make the package.sh easier to use locally.

Before we started, the packaging script only worked in Continuous Integration. I modified some paths to look somewhere only when CI runs. Remembering how to do Bash if statements is surprisingly difficult for me.

if[-n"${CI}"];then
VENV=venv/bin/
fi

To create a new version of the package to test, we ran:

$ rm-f conductor-abcd.pyz &&CIRCLE_SHA1=abcd package.sh

By chaining the commands together, it's very fast to fetch that command from history and run all the steps in a single line.

With our testable package available, we started Celery with a variety of options. When I finished, we settled on an invocation that looked like: