HTTPS clone URL

Subversion checkout URL

Home

Pages 4

Clone this wiki locally

0MQ plugin for RabbitMQ

The purpose of this plugin is to provision 0MQ sockets that relay messages into RabbitMQ. Each 0MQ socket type is given an idiomatic encoding in the AMQP broker model, which means AMQP clients can interoperate.

Among other things, this means you can use RabbitMQ as a device in a 0MQ network, gaining rabbity goodness such as persistence and monitoring. You can use 0MQ as an interconnect to create federations of RabbitMQ brokers. RabbitMQ can serve as a gateway from 0MQ to other protocols such as AMQP, STOMP or XMPP. You can also use different 0MQ language bindings to access your RabbitMQ server from languages/platforms where there are no AMQP clients available.

Prerequisites

You have to have both RabbitMQ server, 0MQ and Erlang binding for 0MQ installed.

Each service entry in the configuration is a triple consisting of a “rendez-vous point”, specification of what type of 0MQ sockets are going to connect to that rendez-vous point and the 0MQ-style address to bind the rendez-vous point to.

Rendez-vous point corresponds either to AMQP exchange or AMQP queue, depending on the messaging pattern. For req/rep pattern it’s the name of the request queue. For pub/sub pattern it’s the name of the exchange that serves as an entry point for the published messages. For pipeline (push/pull) pattern it corresponds to the shared queue to pass messages through.

For detailed diagrams of AMQP wiring corresponding to individual messaging patterns have a look at dedicated pages:

It’s not common for a broker to actively connect to another node on the network, however, it may come handy in federation scenarios. In such cases 0MQ address in the triple can be replaced by a pair consisting of bind or connect keyword and the 0MQ-style address:

reqrep: This example illustrates how to do RPC-style communication. rep.py is the service. It waits for a message from the client and responds with “World!” message. rep.py is the client. It sends “Hello!” message and prints out the reply. You can run as many services and clients in parallel as you wish.

pipeline: This example shows how RabbitMQ broker can be used to gather messages from one processing step in the pipeline and forward them to the next step of the pipeline. push.py represents the previous step in the pipeline. It sends one message per second the the broker. pull.py represents the next processing step. It reads messages from the broker and prints them out. You can run as many pushers and pullers in parallel as you wish. The messages are load-balanced between individual pullers.

pubsub: This example shows how to distribute messages to multiple clients. pub.py is the publisher. It sends one message per second. sub.py is the subscriber. It subscribes for the messages and prints them out. You can run as many publishers and subscribers in parallel as you wish.

Additionally, each example contains Java programs witch implement exactly the same functionality. These are true AMQP clients and you’ll need to install RabbitMQ Java Client to be able to compile and run them:

Known limitations

Lack of socket backpressure

Currently we use “active” mode for erlzmq sockets, whereby 0MQ messages are delivered straight to Erlang process mailboxes. This means that sending processes can happily send a million messages a second and fill available memory at the broker. Ideally we would propagate backpressure to the 0MQ socket so that RabbitMQ can make sure messages, if they need to rest at the broker, are safely in queues. We’re working on fixing this limitation.

Message persistence off by default

We default to using AMQP’s delivery-mode=1, i.e., non-persistent messages. This means they won’t be written to disk if they can’t be delivered straight away. We’re going to make it an option in the config to turn persistence on for a particular socket.

Only AMQP message bodies are sent between brokers

When using the bridge as a means of federation between Rabbits, only the message body is sent over 0MQ; at the other end, it’s reconstituted with defaults (so it will lose headers, delivery mode, and so on). We’re thinking of adding an option for “whole message” serialisation — the bridge would serialise and deserialise AMQP messages to and from the 0MQ payloads.