How to Use Background Tasks in Rails

How to Use Background Tasks in Rails

Why?

Rails servers have a limited amount of connections that they can maintain at one time because the web server (in most cases) is a single-thread and multi-processed. There is a set amount of “workers” that can handle connections, when those connections are tied up, any new connections will have to wait. If you are processing potentially long tasks within the request/response cycle, you might have users waiting for those tasks to finish before they are able to load your website. HTTP is designed to be very fast and a request/response cycle should happen as fast as possible.

Common use-cases

You are connecting to a third-party API and don’t need the response immediately (maybe they have a callback service to process responses)

You have a long running export, for example you need to generate a CSV report of some kind

You are sending e-mails

How

Rails has a ton of gems that will handle backgrounding for you, the most popular are Resque and Sidekiq. In my opinion, Sidekiq is the better option because it is more actively maintained and it uses less memory. That being said, you will need to use Resque if your jobs are not threadsafe (which is unlikely). Both Resque and Sidekiq will be automatically tracked by NewRelic, so you can see failed job logs in the NewRelic console.

I will use Sidekiq in examples but Resque is similar, there are just some slight syntax changes.

A worker is a plain Ruby class that responds to #perform. From that method you can do whatever you need to do and all of it will be executed in the background:

classMyWorker

include Sidekiq::Worker

def perform(user_id)

user = User.find(user_id)

send_email_to(user.email)

end

end

It is important to note that the parameters sent to a worker should be a simple JSON datatype: string, integer, float, boolean, null, array or hash due to the way they store the parameters in Redis. Storing an instance of an object, such as user is a bad idea, since that instance might be stale by the time the worker actually runs.

If you are on a version of Rails < 4.2, I think it is best to use a small abstraction in case you ever switch background job gems. I use something like this:

classBackground

class<< self

def push(worker, *arguments)

Sidekiq::Client.enqueue(worker, *arguments)

end

end

end

Now if you switch to Resque, you only need to change this class. On Rails >= 4.2, Rails has a tool called ActiveJob built-in that provides a similar abstraction.

Recurring Jobs with Sidetiq

Sidetiq is a recurring job processor that is built for Sidekiq. It uses IceCube’s DSL to determine how often a worker should run, and it has a built-in web interface to track and manually trigger recurring jobs. In my opinion, it is cleaner and easier to use Sidekiq+Sidetiq instead of resorting to lower-level cron like whenever. Your recurring jobs can fit in with other background workers you might have, and you gain the the web interface for free.

How

Requirements

The requirements documentation are incomplete/missing from the Sidetiq repository. There are a couple of things you need to be sure of:

1. Redis > 2.6. As I’m writing this, Redis stable is 2.8.7, so that should be no problem. You can check the redis version on your server by running redis-cli INFO, it will be at the top. If you need to update Redis, make sure you flush your old cache keys and redeploy your application.

sudo add-apt-repository ppa:chris-lea/redis-server

sudo apt-get update

sudo apt-get install redis-server

2. config.eager_load should be true in your Rails environment. This is configurable in your environment config file (staging.rb, production.rb, etc).