DESCRIPTION

This is experimental software. Interfaces and implementation are subject to change. If you are interested in using this in production, please get in touch to gauge the current state of stability.

This module implements an abstract interface to a sharded key-value store. The storage backends as well as the "continuum" are pluggable. "Continuum" is to mean "the logic that decides in which shard a particular key lives". Typically, people use consistent hashing for this purpose and very commonly the choice is to use ketama specifically. See below for references.

Beside the abstract querying interface, this module also implements logic to add one or more servers to the continuum and use passive key migration to extend capacity without downtime. Do make it a point to understand the logic before using it. More on that below.

PUBLIC ATTRIBUTES

continuum

The continuum object decides on which shard a given key lives. This is required for a ShardedKV object and must be an object that implements the ShardedKV::Continuum role.

migration_continuum

This is a second continuum object that has additional shards configured. If this is set, a passive key migration is in effect. See begin_migration below!

storages

A hashref of storage objects, each of which represents one shard. Keys in the hash must be the same labels/shard names that are used in the continuum. Each storage object must implement the ShardedKV::Storage role.

PUBLIC METHODS

get

Given a key, fetches the value for that key from the correct shard and returns that value or undef on failure.

Different storage backends may return a reference to the value instead. For example, the Redis and Memory backends return scalar references, whereas the mysql backend returns an array reference. This might still change, likely, all backends may be required to return scalar references in the future.

set

Given a key and a value, saves the value into the key within the correct shard.

The value needs to be a reference of the same type that would be returned by the storage backend when calling get(). See the discussion above.

delete

Given a key, deletes the key's entry from the correct shard.

In a migration situation, this might attempt to delete the key from multiple shards, see below.

begin_migration

Given a ShardedKV::Continuum object, this sets the migration_continuum property of the ShardedKV, thus beginning a passive key migration. Right now, the only kind of migration that is supported is adding shards! Only one migration may be in effect at a time. The passive qualification there is very significant. If you are, for example, using the Redis storage backend with a key expiration of one hour, then you know, that after letting the passive migration run for one hour, all keys that are still relevant will have been migrated (or expired if they were not relevant).

If there is a migration continuum, then for get requests, that continuum is used to find the right shard for the given key. If that shard does not have the key, we check the original continuum and if that points the key at a different shard, we query that.

For delete requests, we also attempt to delete from the shard pointed to by the migration continuum AND the shard pointed to by the main continuum.

For set requests, we always only use the shard deduced from the migration continuum

end_migration() will promote the migration continuum to the regular continuum and set the migration_continuum property to undef.