[Part 1] – Use beanstalkd messaging queue with tomatocart

Behind the scenes, Tomatocart has been developed in a server side script language called PHP. It is great at performing the application tasks synchronously.

For every order that you get, TomatoCart needs to perform the following:

The new order will be inserted into the database.

A new order created email will be sent to the customer.

The second step will only be performed after completing the first step. It maybe very slow to connect to a SMTP server to send the email due to a bad network. The checkout page for customer is blocked until the email is sent. This is really bad.

We need to find a way to perform the above steps asynchronously. When the customer clicks the order confirm button, the system will just need to insert the order into the database. For sending the email to a customer, it will be performed in the background. So, the create new order action will be completed quickly from customer’s point of view.

Beanstalkd

There is a simpler, reliable and effective messaging queue tool named beanstalkd. Some exemplary use-cases for Banstalkd are:

Performing asynchronous jobs to the backend systems.

Distributing a job to multiple workers for processing.

Setting different priority for each task to handle orderly from high to low.

Beanstalkd Concept

Job – a task need to be performed asynchronously. It’s a basic element pushed into a specific tube.

Tube – a named queue to store specific type of tasks. It’s the element operated by producer and consumer.

Producer – Put the job into a specific tube by calling put command.

Worker – Get the job or change the job status by calling reserve/release/bury/delete command and update the database.

Following image describe the life time of a job within beanstalkd:

A job has READY, RESERVED, DELAYED, BURIED four status.

When a producer put a job into a specific tube, it will become READY status. Only the job with READY STATUS could be reserved by worker.

If put the job with delay, it will become DELAYED status. It could be updated to READY after the time passes.

Once the job is reserved by worker, it will be marked with RESERVED.

After worked performed the job completely, the job could be deleted or buried.

If the job is buried by worked, it could be kicked to READY at a later time.

Installing Beanstalkd on Ubuntu

Run the following command to download and install Beanstalkd:

sudo apt-get install -y beanstalkd

Edit the default configuration with vim:

sudo vim /etc/default/beanstalkd

Beanstalkd options

Now, you can start working with the Beanstalkd server. Here are the options for running the daemon:

-b DIR wal directory
-f MS fsync at most once every MS milliseconds (use -f0 for "always fsync")
-F never fsync (default)
-l ADDR listen on address (default is 0.0.0.0)
-p PORT listen on port (default is 11300)
-u USER become user and group
-z BYTES set the maximum job size in bytes (default is 65535)
-s BYTES set the size of each wal file (default is 10485760)
(will be rounded up to a multiple of 512 bytes)
-c compact the binlog (default)
-n do not compact the binlog
-v show version information
-V increase verbosity
-h show this help

Example Usage:

beanstalkd -l 127.0.0.1 -p 11301 &

You could start, stop, restart beanstalkd server:

# To start the service:
service beanstalkd start
# To stop the service:
service beanstalkd stop
# To restart the service:
service beanstalkd restart
# To check the status:
service beanstalkd status

Find word

In this article, i introduce you what’s the beanstalkd, how it works and how to install and manage it. In next article, i will teach you how to use it with TomatoCart system.