PHP SimpleQueue

Overview

PHP SimpleQueue is a PHP/MySql based generic queuing mechanism. If you have ever needed to be able
to manage tasks by queuing them up and processing them all at once, or if you have ever had a use
for a throttling mechanism instead of processing everything at request time, this queue may be just
what you're looking for.

By keeping the queue very generic and lightweight, it can be used to house as many different types
of events as needed, each in its own context. Then, with an appropriately indexed MySql table, specific
contexts can be queried for outstanding queued items very quickly.

Additionally, PHP SimpleQueue has a built in retry mechanism that will allow you to define both
the number of retries for processing a specific item as well as how long to delay between retries.

PHP SimpleQueue has a self installer, so you simply need to FTP the package to your server,
launch the install.php script from your browser, follow the prompts for configuration, and you will
be able to use the queue from any of your scripts.

Note

PHP SimpleQueue is packaged with a logging class on which it depends to function. By default, it
will write to the current directory which the script is running. If you define a LOG_PATH
variable, this path will be used instead.

PHP SimpleQueue also checks to see if you have defined a DEBUG variable. If so, and
it is set to true, verbose logging messages will be posted to the log file for your perusal.

For testing purposes, you could simply turn debugging on and write to your /tmp
directory by adding the following two lines to your script:

<?phpdefine('DEBUG', TRUE);define('LOG_PATH', '/tmp');?>

This would result in verbose logging being dumped to /tmp/SimpleQueue.log.

Example

Let's create a basic scenario: you have a webpage that allows users to contact a 3rd party API
(let's say Twitter). As traffic increases to your site, you realize you don't want every request
to be processed immediately and tie up resources, but you'd rather throttle the requests a few at a
time.

The first step is to simply queue the requests that come in and give the user a successful page
load. This gives them a great user experience, and it also helps you spread your server load more
evenly. Your queuing code could be as simple as this:

<?php// Optional context is "twitter" for this example// $request can be any data you need to processrequire('lib/PHPSimpleQueue.php');$q = new SimpleQueue();$q->queue(json_encode($request), 'twitter');?>

Now that you have things queued up, we need to define a callback function to process the request and
tell PHP SimpleQueue whether or not each item was successfully processed. If not, it will be requeued
according to the configs, otherwise, it will be permanently deleted from the queue.

We need a script that will run on a CRON job (let's say every 5 minutes). We know we want to process
25 twitter queue items every 5 minutes, so our script could be as simple as this:

// Set up queue rulesrequire('lib/PHPSimpleQueue.php');$q = new SimpleQueue();$q->setExecLimit(25); // 25 per run$q->setRetryLimit(3); // After 3 failures, remove from the queue$q->setRetryDelay(15); // Wait 15 minutes between retries$q->setCallbackAction('processTwitter', 'twitter'); // Set callback for "twitter" context$q->dequeue('twitter'); // Tell the queue to process this context based on configs?>