A Must Have Redis Patch

15 Jul

(or “how to get the whole hash with SORT…BY…GET)

Suppose you’ve built Reddit on top of Redis. Using Redis’ built in hash data structure, you might store a news item under the key “items:78” where 78 is the unique id of that news item:

“items:78”: { id: 78, title: “My Blog”, url: “https://ngchi.wordpress.com”}

You would then store your home page as a Redis list data structure of these unique ids under the key “home.page”:

“home.page”: [125, 132, 143, 113, 78]   // References the top 5 links

From your application code, you’ll want an array of the home page’s top 5 articles in the following easy-consumption format:

[ { id: 125, title: “Redis is an awesome database”, url: “http://code.google.com/p/redis” }, …, { id: 78, title: “My Blog”, url: “https://ngchi.wordpress.com” } ]

With the current Redis release, you’ll have to invoke 3 different commands to retrieve the id, title, and url data respectively, and then you’ll have to arrange them into the desired array at the application level:

SORT home.page BY nosort GET items:*->id

SORT home.page BY nosort GET items:*->title

SORT home.page BY nosort GET items:*->title

For hashes that each have N different key/value pairs, this means we need to invoke N commands to grab N lists of items! That’s unnecessary work. All we really want is 1 command to grab 1 list of items, each with N different key/value pairs. We can do better.

Hence, I patched Redis so that it does in 1 command what it took 3 commands above to do. The command we all want is now thus:

SORT home.page BY nosort GET items:*

This was a single patch to the Redis server code. But we’ll also need to patch the client code to interpret this new interpretation of the Redis protocol when using SORT. With the right client implementation, we can retrieve the desired array of hashes from our application. I therefore also made a corresponding patch in Ezra’s redis-rb Ruby Redis client.

You can give the new capability a try by downloading my patched redis branch and patched redis-rb (Redis ruby client) branch here:

See this gist for more details.

I’ll be making pull requests to have these patches integrated into the main branch of Redis and then redis-rb. As a disclaimer, this is my first time hacking a large C project in a while, so use at your own risk, and let me know if you uncover any bugs/memory leaks that my patch causes.


One Response to “A Must Have Redis Patch”

  1. april coin May 28, 2015 at 12:00 pm #

    Thanks for writing this good idea

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: