AMQP, RabbitMQ and Celery - A Visual Guide For Dummies

Celery is an asynchronous distributed task queue. RabbitMQ is a message broker which implements the Advanced Message Queuing Protocol (AMQP). Before we describe relationship between RabbitMQ and Celery, a quick overview of AMQP will be helpful [1][2].

Amqp Key Terms

Message Or Task

A message or task consists of attributes (headers) and payload (body). Some attributes are used by broker but most are used by consumers. Optional attributes are known as headers. Some common attributes are

Content type

Content encoding

Routing key

Massage payload or body contains data that goes to the consumer. Normally a serialisation format such as JSON is used for message payload. Content type and content encoding attributes are used to communicate the serialisation format. An example message payload serialised in JSON looks like,

Producer

A producer is a user application that sends messages.

Broker

A broker receives messages from producer and router them to consume. A broker consists an exchange and one or more queues.

Exchange

A producer can send messages to queues only via exchange. Exchanges take a message from producer and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings.

Queue

A message or task queue is a buffer that stores messages.

Bindings

Bindings are rules that the exchange uses to route messages to queues.

Routing Keys

Bindings may have an optional routing key attribute. An exchange may use this field to route a message to the bound queue.

Consumer

A consumer is an application that receives messages and process them.

Celery

Celery generally hides the complexity of AMQP protocols [3]. Celery act as both the producer and consumer of RabbitMQ messages. In Celery, the producer is called client or publisher and consumers are called as workers. It is possible to use a different custom consumer (worker) or producer (client).