Monitoring Laravel’s background queues in realtime using Pusher

Published Jan 19, 2018

When building large applications, making it scale is usually a concern. Stats like how long it takes for the page to load is usually very important. Thus, doing things like processing large images, sending emails and SMS can be pushed to a background queue and processed at a later time.

However, because queues work in the background, they can fail sometimes. It may then be necessary to be able to monitor background queues.

In this article, we will consider ways to monitor Laravel’s background queues in realtime. We will assume we created an application that sends emails. The emails are to be queued in the background and sent later. We will then have a report page with the emails that have been sent and those that haven’t.

This will generate an authentication scaffold. That is all that you need to do regarding authentication.
Configuring Pusher
Replace the PUSHER_* keys in the .env file with the correct keys you got from your Pusher dashboard:

Configuring Other Miscellaneous Things
Open the .env file and change the BROADCAST_DRIVER to pusher, and the QUEUE_DRIVER to database. To make sure we have the tables necessary to use database as our QUEUE_DRIVER run the command below to generate the database migration:

$ php artisan queue:table

Then run the migrate command to migrate the database:

$ php artisan migrate

This will create the database table required to use our database as a queue driver.

TIP: In a production environment, it is better to use an in-memory storage like Redis or Memcached as the queue driver. In-memory storage is faster and thus has better performance than using a relational database.

Building the Backend of Our Application

Now let’s create the backend of our application. Run the command below in your terminal:

$ php artisan make:model QueuedEmails

This will create a new model in the app directory. Open the file and replace the contents with the following:

In the code above, we define the fillable property of the class. This will prevent a mass assignment exception error when we try to create a new entry to the database. We also specify a casts property which will instruct Eloquent to typecast attributes to data types.

Next, open the HomeController and and replace the contents with the code below:

In the controller above, we have 4 methods that are mostly self-explanatory. In the class we use the Faker library which helps us generate random fake values. In the simulate method, we are using the faker library to generate a fake email address and description. We instantiate a SimulateMailmailable.

The noteworthy aspect of the code above is the jobs tag. This is a reference to the Vue component we will create next. We also have a “Simulate” button that leads to a /simulate route. This route simulates queuing an email to be sent.

Open your terminal and type in the command below:

$ npm install --save laravel-echo pusher-js

This will install Laravel Echo and the Pusher JS SDK. When the installation is complete, run the command below to install the other NPM dependencies:

$ npm install

Building our Vue Component
Let’s build the jobs Vue component we referenced earlier. Open the resources/assets/js/app.js file and replace the code below:

In the Vue component above, we have defined a template. In there, we loop through the jobs array and list each job’s description and timestamp.

In the created method of the Vue component script, we have a refreshAllJobs function that uses Axios (a HTTP request library built-in Laravel by default) to make a request to the /jobs route. We then assign the response to the allJobs property.

In the same method, we use Laravel Echo to listen to a Pusher channel and wait for an event to be triggered. Whenever the events .add and .sent are triggered, we call the refreshAllJobs method.

TIP:* The event names have a dot before them because, in Laravel, whenever you use thebroadcastAs method to define an alias you need to add the dot. Without the dot your event will not be caught by the listener. If you do not provide an alias, Laravel will use the namespace + class as the name of the broadcast event.

Open the resources/assets/js/bootstrap.js file. At the bottom of the file, add the following code:

NOTE: Make sure you replace the PUSHER_APP_KEY and PUSHER_APP_CLUSTER with your Pusher application key and cluster.

Finally, run the command below to build your assets:

$ npm run dev

Testing Our Application

After the build is complete, start a PHP server if you have not already by running the command below:

$ php artisan serve

This will create a PHP server so we can preview our application. The URL will be provided on the terminal but the default is http://127.0.0.1:8000.

When you see the Laravel homepage, create a new account using the ”Register” link on the top right corner of the page. Now click the “Simulate” button and you should see a new queued email entry.

Now we will manually execute the processes on our queue using the queue:listen artisan command. Open a new terminal window and run the command below:

$ php artisan queue:listen

This should start executing any queues it sees. As long as the terminal is open and the queue:listen command is running, when you click the “Simulate” button the queue will run immediately. If you kill the queue:listen command, the queue entries will remain there and not be triggered.

TIP: In a production environment, you cannot keep**queue:listenrunning and you might need a worker running on a background process; something likeSupervisor**. You can read more about how you can do that** here.

Conclusion

In this article, we have been able to create a realtime Laravel queue monitor using Pusher and Vue. Having queues that you can track and quantify can be useful. Hopefully, you picked something from this article. If you have any questions or feedback, feel free to ask in the comments section.