An updated version of Redis has been released with new features including a new data structure and extra commands for sorted data sets.

Redis is becoming increasingly popular as an open source advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets.

Its main developer and creator, Salvatore Sanfilippo, worked at VMWare for some time, and the project is now supported by Pivotal Software which hosts it on the open source platform as a service (PaaS) Cloud Foundry, which was developed, and is still funded by VMware together with EMC..

The most recent release, Redis 2.8.9, has added new features and commands, but no bug fixes. Sanfilippo says this is because there were no bugs to fix, showing that the 2.8 branch is starting to become pretty stable.

The new feature is a new data structure called the HyperLogLog. This is a hash technique for computing the number of distinct entries in a set using only a small amount of memory. There’s a useful and enthusiastic explanation of HyperLogLogs on the Antirez blog. In his announcement of Redis 2.8.9 on Google Groups, Sanfilippo explains his implementation supports the sparse representation of HyperLogLogs, meaning that means that HLLs with low cardinalities of less than 4000 don't need to use 12k of memory, but a lot less. He says for lower cardinalities, the amount of memory saved is substantial, and in cases where you have many HLLs counting different things with a long-tail distribution, this really makes a difference between the ability to apply Redis or not.

The new release also adds three new commands for the sorted data set structure to support lexicographical range queries. The commands are ZRANGEBYLEX, ZLEXCOUNT, and ZREMRANGEBYLEX. Lexicographical range queries in Redis make it useful for applications such as building secondary indexes, or using it as a high performance autocompletion server. Sanfilippo has a demo showing this in action. He says that using sorted sets features you can mount an incremental autocompletion engine with a few lines of code, giving the examples of using ZINCRBY to build a sorted set showing query popularity.

For the next few months, development work will be focused on Redis Cluster and Sentinel, but Sanfilippo says Redis 2.8.9 sends a signal to the Redis community that while Redis Cluster will be a feature (like Redis persistence or replication are) the Redis project main goal is to provide a core of easy to exploit in-memory data structures.

AI and robotics have been making headline-grabbing progress recently and high profile people have started to worry in public about the future. Now we have over 1000 signatures on a letter urging a ban [ ... ]