Introduction

The Laravel queue service provides a unified API across a variety of different queue back-ends. Queues allow you to defer the processing of a time consuming task, such as sending an e-mail, until a later time which drastically speeds up web requests to your application.

Configuration

The queue configuration file is stored in config/queue.php. In this file you will find connection configurations for each of the queue drivers that are included with the framework, which includes a database, Beanstalkd, IronMQ, Amazon SQS, Redis, and synchronous (for local use) driver.

A null queue driver is also included which simply discards queued jobs.

Driver Prerequisites

Database

In order to use the database queue driver, you will need a database table to hold the jobs. To generate a migration that creates this table, run the queue:table Artisan command. Once the migration is created, you may migrate your database using the migrate command:

php artisan queue:table
php artisan migrate

Other Queue Dependencies

The following dependencies are needed for the listed queue drivers:

Amazon SQS: aws/aws-sdk-php ~3.0

Beanstalkd: pda/pheanstalk ~3.0

IronMQ: iron-io/iron_mq ~2.0

Redis: predis/predis ~1.0

Writing Job Classes

Generating Job Classes

By default, all of the queueable jobs for your application are stored in the app/Jobs directory. You may generate a new queued job using the Artisan CLI:

php artisan make:job SendReminderEmail --queued

This command will generate a new class in the app/Jobs directory, and the class will implement the Illuminate\Contracts\Queue\ShouldQueue interface, indicating to Laravel that the job should be pushed onto the queue instead of run synchronously.

Job Class Structure

Job classes are very simple, normally containing only a handle method which is called when the job is processed by the queue. To get started, let's take a look at an example job class:

In this example, note that we were able to pass an Eloquent model directly into the queued job's constructor. Because of the SerializesModels trait that the job is using, Eloquent models will be gracefully serialized and unserialized when the job is processing. If your queued job accepts an Eloquent model in its constructor, only the identifier for the model will be serialized onto the queue. When the job is actually handled, the queue system will automatically re-retrieve the full model instance from the database. It's all totally transparent to your application and prevents issues that can arise from serializing full Eloquent model instances.

The handle method is called when the job is processed by the queue. Note that we are able to type-hint dependencies on the handle method of the job. The Laravel service container automatically injects these dependencies.

When Things Go Wrong

If an exception is thrown while the job is being processed, it will automatically be released back onto the queue so it may be attempted again. The job will continue to be released until it has been attempted the maximum number of times allowed by your application. The number of maximum attempts is defined by the --tries switch used on the queue:listen or queue:work Artisan jobs. More information on running the queue listener can be found below.

Manually Releasing Jobs

If you would like to release the job manually, the InteractsWithQueue trait, which is already included in your generated job class, provides access to the queue job release method. The release method accepts one argument: the number of seconds you wish to wait until the job is made available again:

Checking The Number Of Run Attempts

As noted above, if an exception occurs while the job is being processed, it will automatically be released back onto the queue. You may check the number of attempts that have been made to run the job using the attempts method:

Pushing Jobs Onto The Queue

The default Laravel controller located in app/Http/Controllers/Controller.php uses a DispatchesJobs trait. This trait provides several methods allowing you to conveniently push jobs onto the queue, such as the dispatch method:

Of course, sometimes you may wish to dispatch a job from somewhere in your application besides a route or controller. For that reason, you can include the DispatchesJobs trait on any of the classes in your application to gain access to its various dispatch methods. For example, here is a sample class that uses the trait:

Specifying The Queue For A Job

You may also specify the queue a job should be sent to.

By pushing jobs to different queues, you may "categorize" your queued jobs, and even prioritize how many workers you assign to various queues. This does not push jobs to different queue "connections" as defined by your queue configuration file, but only to specific queues within a single connection. To specify the queue, use the onQueue method on the job instance. The onQueue method is provided by the base App\Jobs\Job class included with Laravel:

Delayed Jobs

Sometimes you may wish to delay the execution of a queued job. For instance, you may wish to queue a job that sends a customer a reminder e-mail 15 minutes after sign-up. You may accomplish this using the delay method on your job class, which is provided by the Illuminate\Bus\Queueable trait:

In this example, we're specifying that the job should be delayed in the queue for 60 seconds before being made available to workers.

Note: The Amazon SQS service has a maximum delay time of 15 minutes.

Dispatching Jobs From Requests

It is very common to map HTTP request variables into jobs. So, instead of forcing you to do this manually for each request, Laravel provides some helper methods to make it a cinch. Let's take a look at the dispatchFrom method available on the DispatchesJobs trait. By default, this trait is included on the base Laravel controller class:

This method will examine the constructor of the given job class and extract variables from the HTTP request (or any other ArrayAccess object) to fill the needed constructor parameters of the job. So, if our job class accepts a productId variable in its constructor, the job bus will attempt to pull the productId parameter from the HTTP request.

You may also pass an array as the third argument to the dispatchFrom method. This array will be used to fill any constructor parameters that are not available on the request:

Running The Queue Listener

Starting The Queue Listener

Laravel includes an Artisan command that will run new jobs as they are pushed onto the queue. You may run the listener using the queue:listen command:

php artisan queue:listen

You may also specify which queue connection the listener should utilize:

php artisan queue:listen connection

Note that once this task has started, it will continue to run until it is manually stopped. You may use a process monitor such as Supervisor to ensure that the queue listener does not stop running.

Queue Priorities

You may pass a comma-delimited list of queue connections to the listen job to set queue priorities:

php artisan queue:listen --queue=high,low

In this example, jobs on the high queue will always be processed before moving onto jobs from the low queue.

Specifying The Job Timeout Parameter

You may also set the length of time (in seconds) each job should be allowed to run:

php artisan queue:listen --timeout=60

Specifying Queue Sleep Duration

In addition, you may specify the number of seconds to wait before polling for new jobs:

php artisan queue:listen --sleep=5

Note that the queue only "sleeps" if no jobs are on the queue. If more jobs are available, the queue will continue to work them without sleeping.

Supervisor Configuration

Supervisor is a process monitor for the Linux operating system, and will automatically restart your queue:listen or queue:work commands if they fail. To install Supervisor on Ubuntu, you may use the following command:

sudo apt-get install supervisor

Supervisor configuration files are typically stored in the /etc/supervisor/conf.d directory. Within this directory, you may create any number of configuration files that instruct supervisor how your processes should be monitored. For example, let's create a laravel-worker.conf file that starts and monitors a queue:work process:

In this example, the numprocs directive will instruct Supervisor to run 8 queue:work processes and monitor all of them, automatically restarting them if they fail. Once the configuration file has been created, you may update the Supervisor configuration and start the processes using the following commands:

For more information on configuring and using Supervisor, consult the Supervisor documentation. Alternatively, you may use Laravel Forge to automatically configure and manage your Supervisor configuration from a convenient web interface.

Daemon Queue Listener

The queue:work Artisan command includes a --daemon option for forcing the queue worker to continue processing jobs without ever re-booting the framework. This results in a significant reduction of CPU usage when compared to the queue:listen command:

As you can see, the queue:work job supports most of the same options available to queue:listen. You may use the php artisan help queue:work job to view all of the available options.

Coding Considerations For Daemon Queue Listeners

Daemon queue workers do not restart the framework before processing each job. Therefore, you should be careful to free any heavy resources before your job finishes. For example, if you are doing image manipulation with the GD library, you should free the memory with imagedestroy when you are done.

Similarly, your database connection may disconnect when being used by long-running daemon. You may use the DB::reconnect method to ensure you have a fresh connection.

Deploying With Daemon Queue Listeners

Since daemon queue workers are long-lived processes, they will not pick up changes in your code without being restarted. So, the simplest way to deploy an application using daemon queue workers is to restart the workers during your deployment script. You may gracefully restart all of the workers by including the following command in your deployment script:

php artisan queue:restart

This command will gracefully instruct all queue workers to restart after they finish processing their current job so that no existing jobs are lost.

Note: This command relies on the cache system to schedule the restart. By default, APCu does not work for CLI jobs. If you are using APCu, add apc.enable_cli=1 to your APCu configuration.

Dealing With Failed Jobs

Since things don't always go as planned, sometimes your queued jobs will fail. Don't worry, it happens to the best of us! Laravel includes a convenient way to specify the maximum number of times a job should be attempted. After a job has exceeded this amount of attempts, it will be inserted into a failed_jobs table. The name of the failed jobs can be configured via the config/queue.php configuration file.

To create a migration for the failed_jobs table, you may use the queue:failed-table command:

php artisan queue:failed-table

When running your queue listener, you may specify the maximum number of times a job should be attempted using the --tries switch on the queue:listen command:

php artisan queue:listen connection-name --tries=3

Failed Job Events

If you would like to register an event that will be called when a queued job fails, you may use the Queue::failing method. This event is a great opportunity to notify your team via e-mail or HipChat. For example, we may attach a callback to this event from the AppServiceProvider that is included with Laravel:

Retrying Failed Jobs

To view all of your failed jobs that have been inserted into your failed_jobs database table, you may use the queue:failed Artisan command:

php artisan queue:failed

The queue:failed command will list the job ID, connection, queue, and failure time. The job ID may be used to retry the failed job. For instance, to retry a failed job that has an ID of 5, the following command should be issued:

php artisan queue:retry 5

If you would like to delete a failed job, you may use the queue:forget command:

php artisan queue:forget 5

To delete all of your failed jobs, you may use the queue:flush command: