Running Background Jobs in Ruby on Rails Revisited

The Ruby on Rails framework has a number of tools for running your code outside of the web-request, including the venerable script/runner for one-off tasks, but using them can be a little heavy on your server. If you want to run a task on the minute, or on demand, script/runner will load your entire Rails environment, which can be from 20-50 MB, depending on how many libraries and how much code you’re pulling in.

There are also a few other good guides, recipes, and libraries that we’ve mentioned before, including:

A chapter I helped write in Obie Fernandez’s The Rails Way on Background Processing

We’ve found that it’s not terribly hard to build your own job server that runs continuously in the background and can handle all kinds of jobs, including those that should run on a specified interval. Here’s how we did it.

We’re going to make use of the Daemons gem, so install it first:

sudo gem install daemons

Let’s go ahead and build in two types of jobs:

those that Run Once (immediately) and

those that Run on an interval (every x seconds or minutes or days)

We’ll use ActiveRecord’s Single Table Inheritance (STI) to handle both types of jobs and dictate their differing behaviors.

Now, we have a built in system for running Periodic Jobs. Note that all we have to do is create a new Periodic Job with the actual code we would normally toss to script/runner in the PeriodicJob#code field, and when we call the PeriodicJob#run! method, it will evaluate it.

We now need a way to always run a background task server to check these PeriodicJobs and run them.

loopdo# Find all Run Once jobs, and run them RunOncePeriodicJob.find_all_need_to_run.each do |job| job.run! end # Find all Run on Interval jobs, and run them RunIntervalPeriodicJob.find_all_need_to_run.each do |job| job.run! end # Cleans up periodic jobs, removes all RunOncePeriodicJobs over one # day old. RunOncePeriodicJob.cleanup sleep(SLEEP_TIME) end

Note the use of the task_server, so you can simply allow one app server to be your task server (if you’re running on multiple servers).

And now, because I’m feeling generous, let’s set monit up to monitor your task server, so that if it ever goes down for some strange reason, monit should boot it back up (this also ensures that restarts will boot your task server back up):