Not Logged In

distributex 0.6

Distributex

Distributex is a simple mutex service for coordinating certain cluster
operations.

Note: Distributex is not designed for tasks which require high performance
or fair lock acquisition. It is a very simple Busy-waiting lock with very slow
acquisition. Do not use it for extremely large clusters either as there’s
a good chance a requester might never obtain a lock.

Distributex server

The Distributex server provides a simple HTTP service. It is written using
Twisted, and provides a Twisted plugin. It also requires PyYAML for its
configuration.

You can start it as follows, or wrap it in supervisor, or pass -d, or whatever:

$ twistd -n distributex -c distributex.yaml

The configuration file is a simple YAML structure which defines lock pools. A
lock pool is a resource you want to allow things to fight over.:

This will create two pools whose lock expires after 5 minutes. It’s generally
a good idea to set an expiry to ensure something, otherwise it will default to
30 minutes. If you don’t want it to expire then set it to 0, but I don’t
recommend that.

You can specify either the ‘memcache’ backend or ‘inmemory’, there are pros
and cons to both. Memcache will be slower, but state is retained away from
the Distributex server and you can scale out workers - however since the
inmemory backend can handle about 5000 waiting locks on a single machine
redundancy is the only real concern. Lock expiry is also a bit more reliable
and simpler in the memcache backend.

A comma separated list of servers can also be provided to prevent accidental
pool incursions. This isn’t secure, nor are lock releases, since anyone can
just forge their hostname. The distributex client will pass the FQDN of the
host.

It is also possible to set ‘maxlocks’ which allows the pool to behave like
a semaphore.