We're creating a high-throughput AMQP client using Qpid as the broker, and we are getting nowhere near the throughput reported in other places. (We're receiving under 250 msg/s, while this blog post gets over 4 times that.) I fear the weak performance might be due to the fact that Qpid Python client libraries are in pure Python. Does anyone know of a compiled Python AMQP library that is compatible with Qpid 0.7?

Edit 2011-07-01: Although I've accepted zeekay's answer, for Qpid they recently released SWIG bindings for their library. I've made patches that make it a (near) drop-in replacement for their pure Python client libraries. After configuration changes which got us up to about 1000 msgs/sec, using the SWIG version got us to about 7000 msgs/sec. Hopefully other Qpid users will find the bindings (and my patches) useful.

This question appears to be off-topic. The users who voted to close gave this specific reason:

"Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it." – gnat, ProgramFOX, 61612, bummi, Paul Crovella

3 Answers
3

That is extremely slow. Something is definitely wrong, and a different library probably won't make much of a difference. A simple test using rabbitmq and the excellent kombu lib (which has an absolutely lovely API!):

I seem to be able to send and recieve messages at around 7k msg/sec. Kombu can use both py-amqp and pika for synch/asynch clients. As far as clients go you might want to also look into txAMQP for twisted as well. Or even zeromq.

It might be, but I've only learning AMQP for under a month (most of it while doing design away from a computer), have little experience debugging network protocols, and we're supposed to give the client a design in about two weeks. So not by me, and right now. (Plus, I'd rather fork pyamqplib. The API looks cleaner.)
–
AFogliaMay 6 '11 at 17:59