redisfs - A replication-friendly filesystem.

Redis is an in-memory key & value store, which is simple to install, configure & use. It is particularly useful because it is very simple to use in a replicated fashion.

The notion of a replication-friendly storage system inspired me to build a filesystem around it, using the FUSE system.

Using a key & value storage system to store files might not seem like the most sane idea, but it actually works out surprisingly well. Consider the notion that you give each file in your system a unique identifier (literally a single integer) then you can store things such as:

Key

Value

INODE:1:NAME

The name of the file (e.g. "passwd").

INODE:1:SIZE

The size of the file (e.g. "1661" )

INODE:1:GID

The group ID of the file's owner (e.g. "0")

INODE:1:UID

The user ID of the file's owner (e.g. "0")

INODE:1:MODE

The mode of the file (e.g. 0755)

..

..

There is a little additional complexity involved in storing directory entries, such that we can find the entries of a given directory. (We solve that problem by using a Redis "SET".)

Additionally the data associated with each file, which is stored in memory, is compressed via ZLIB to reduce the round-trip time to a potentially remote redis-server.

Replication Options

There are two ways that you can achieve a replicated filesystem with this software; the simple way involves using a single redis server which multiple hosts connect to:

The alternative is to have each filesystem mount itself against a local redis server - using the redis replication system to control the redundancy:

The appeal of this system is that there is less round-trip time to replicate your files; however the drawback is that the filesystem mounted against the slave is by nature read-only.

(Note: If you want to be explicitly read-only that is easily possible; just launch "redisfs --read-only ..".)

Snapshotting

Because all file & directory details are stored in-memory it becomes very trivial to create snapshots. All we have to do is iterate over each of the keys relating to our filesystem - and copy them.

As of redisfs v0.5 redisfs-snapshot allows a filesystem to be snapshotted with ease. Simply execute:

$ redisfs-snapsot --from=skx --to=copy

Note: You don't need root access to snapshot the filesystem, as you're only ever talking to redis.

This takes all keys with a "skx:" prefix (this is the default) and clones them to new keys with a "safe-copy" prefix. You can then mount this filesystem via:

Note: To mount a second filesystem against the same redis server you'll need to configure a key-prefix via the --prefix argument.

Missing Features

The only obvious missing feature is support for hard-links, symlinks are supported though.

After that I'm going to look to see how hard it will be to use redis asynchronously - I suspect that should be simple, but my recollection is that FUSE doesn't play nice with such things - currently I use a mutex for 99% of operations to avoid corruption and inconsistencies.