nydus.db.create_cluster

The create_cluster function is a simple helper to configure a Cluster based on a simple dict config.

There are two required keyword arguments:

backend: full path to the backend class, which should extend nydus.db.backends.BaseConnection.

hosts: a dictionary, where the key is an ordered numeric value, and the result is a dict of connection options.

(the keys are numeric values simply for readability in configuration)

defaults: a dictionary of default settings for all hosts to inherit from.

Optionally, you may also specify a value for router, which is the full path to the router class,
which must extend nydus.db.routers.BaseRouter.

Distributed Queries

In some cases you may want to execute a query on many nodes (in parallel). Nydus has built-in support for this when any routing function
returns a cluster of nodes:

fromnydus.dbimportcreate_cluster>>># by not specifying a router all queries are executed on all hostsredis=create_cluster({'backend':'nydus.db.backends.redis.Redis','hosts':{0:{'db':0},1:{'db':1},2:{'db':2},}})>>># map the query over all connections returned by the default routerres=redis.incr('foo')>>>asserttype(res)==listassertlen(res)==3

You can also map many queries (utilizing an internal queue) over connections (again, returned by the router):

withredis.map()asconn:results=[conn.incr(k)forkinkeys]

As of release 0.5.0, the map() function now supports pipelines, and the included Redis backend will pipeline commands
wherever possible.

Redis

Nydus was originally designed as a toolkit to expand on the usage of Redis at DISQUS. While it does provide
a framework for building clusters that are not Redis, much of the support has gone into providing utilities
for routing and querying on Redis clusters.

You can configure the Redis client for a connection by specifying it’s full path:

Pycassa

Basic connection management for pycassa (Cassandra) clusters is supported, but we use a non-standard syntax
for creating clusters as routing behavior and per-connection options are not useful in this context: