Luke Melia

personal

January 17, 2010

Redis in Practice: Who’s Online?

Redis is one of the most interesting of the NOSQL solutions. It goes beyond a simple key-value store in that keys’ values can be simple strings, but can also be data structures. Redis currently supports lists, sets and sorted sets. This post provides an example of using Redis’ Set data type in a recent feature I implemented for Weplay, our social youth sports site.

The End Result

Weplay members were told us they wanted to be able to see which of their friends were online, so we decided to add the feature. Let’s look at how Redis was instrumental in this functionality.

Properties of a Set

First a quick overview of Redis’s “set” data structure. A set in Redis has the same properties as the abstract data structure of the same name:

* 0 to N elements
* Unordered
* No repeated members

In practice, this is nice because adding a value to a set does not require you to check if the value exists in the set first.

Redis Set Operations

You can see a full list of the set operations Redis supports. here are the ones were going to use:

The Approach

The idea is to have one active set per minute. During each request that comes in from a logged-in user, we’ll add a user ID to the active set. When we want to know which user IDs are online, we can union the last 5 sets to get a collection of user IDs who have made a request in the last 5 minutes.

Now, if we have a set of the user’s friend’s IDs, we can intersect that with the online users and we’ve got our online friend IDs.

The Code

Here’s the relevant code. (Note: we’re using the redis-rb gem, by Ezra Zygmuntowicz (aka @ezmobius), as our Redis client.)

I love that the code is almost as succinct as the diagrams.

With this approach, you need to empty or delete the older sets before the next hour comes around. We use a cron job for that purpose. I’m not going to show that here, but feel free to get in touch if you have any questions about it.

Hope this gives you some ideas of the Redis’ elegant simplicity and power. This code simpler than it would be if we had used an RDBMS or memcached, or both.

If you’re new to Redis, I’d recommend following it’s creator and maintainer, Salvatore Sanfilippo (@antirez). He’s a very thoughtful and transparent open source leader and is rapidly and continually improving Redis.

23 Responses to “Redis in Practice: Who’s Online?”

  1. Twitter Trackbacks for Luke Melia » Redis in Practice: Who’s Online? [lukemelia.com] on Topsy.com chimed in:

    […] Luke Melia » Redis in Practice: Who’s Online? http://www.lukemelia.com/blog/archives/2010/01/17/redis-in-practice-whos-online – view page – cached The personal website of Luke Melia, a vegetarian beach volleyball playing, father, husband and software developer from New York City who attempts to live every day with passion. […]

  2. antirez chimed in:

    Hello, great writeup!

    There is a related pattern, it’s a bit different as it’s not possible to perform intersections but only to test if a given user is online or not, so not applicable for your problem: basically every time an user performs a pageview you SET uid + EXPIRE uid 600 (assuming we want consider it offline after 600 seconds of inactivity). Then you can test if the user is online with EXISTS uid.

    There is also a less precise but faster alternative of your pattern, that is to maintain two sets one “old” and one “new”. Every five minutes to rename the new as old (removing the old at all). All the SADDs are against the new one. The online users are the intersection of new and old sets. This is less precise as an user can be considered online if it was seen in a variable time frame of five to ten minutes.

  3. Tim chimed in:

    brilliant solution! thanks for sharing :)

    tim

  4. Frank chimed in:

    “This code simpler than it would be if we had used an RDBMS or memcached, or both.”

    You don’t imagine how true it is.

    This is what we use at Skyrock and it’s incredibly complex and non-efficient. Using Redis, thanks to SINTER, would have saved weeks of work.

    Another reason for moving to nosql and post-memcache engines.

  5. Aníbal Rojas chimed in:

    Simple, beautiful solution. I have to questions for you A) what’s the size of the sets being unioned / intersected, and B) How much time does this operations take. In my experience with Redis, this operations are _very_ fast, but would like to know your numbers.

  6. lukemelia chimed in:

    @aníbal I’ll pull some benchmarks later and share them. Have to take my daughters to the pool first. :)

  7. Redis in Practice: Who’s Online? « turnings chimed in:

    […] a comment » Redis in Practice: Who’s Online?: Redis is one of the most interesting of the NOSQL solutions. It goes beyond a simple key-value […]

  8. _sunil_ chimed in:

    Hey!

    Nice solution man! Keeping them coming!

  9. mikhailov chimed in:

    Seems, Radis is a great solution among NoSQL friends, by the way Github is using them.
    My last experience is switch Rails.cache and sessions storage to Redis by using redis-store (http://github.com/jodosha/redis-store).
    Redis with RackCache drive me crazy about speed and performance results!
    http://gist.github.com/279839

  10. redis in practice: who’s online? — award tour chimed in:

    […] Luke Melia » Redis in Practice: Who’s Online?. very cool writeup. I had no idea redis could do union operations like that. […]

  11. Ennuyer.net » Blog Archive » Rails Reading January 26 2010 chimed in:

    […] Luke Melia » Redis in Practice: Who’s Online? […]

  12. links for 2010-02-16 « Breyten’s Dev Blog chimed in:

    […] Luke Melia » Redis in Practice: Who’s Online? (tags: redis oneline who is howto implementation) […]

  13. Labnotes » Rounded Corners 245 – Worst. Ideas. Ever. chimed in:

    […] of the Redis API, but once you do … did I mention awesome? Here’s one such example using set operations to report who’s online in real time: The idea is to have one active set per minute. During each request that comes in from a logged-in […]

  14. Jon Binns chimed in:

    Why use a set per minute and do a union instead of just storing a set per five minutes? Or, antirez’s solution of just using a single set, with EXPIRE? Seems like a lot less work, for same amount of win.

    Do elaborate, I’m hoping I missed something.

  15. Stefan Koch chimed in:

    Jon Binns question is pretty old, but in case anybody is stumbling across this article and is waiting for an answer: You would lose accuracy of one minute if you did sets of five minutes. I mean, consider a person had an action at at 12:33 and you have the coice between five sets (30, 31, 32, 33, 34) or one set (30-34). Using five sets and joining the past five minutes, the person with an action at 12:33 would still be displayed online at 12:35, but if you used one single set from 30-34, the person would be displayed offline as of 12:35 because it is not member of the new 5 minutes interval (35-39) anymore. If you used the past two intervals (to circumvent such a problem), you would lose accuracy “backwards”, meaning that a person online at 12:30 would still be displayed online at 12:39.

  16. What The Heck Are You Actually Using NoSQL For? | TouchHappy chimed in:

    […] Calculating whose friends are online using sets. […]

  17. rick hurst chimed in:

    Great write up.
    I use something less elaborate:
    If you have a redis session key (via node.js/express) such that you have something like this sess:n5Zq9HuyiSQJqXjiPDprHoxG.9OvYheqbq9Go1E13VMUQd7c+ZUA+5NXyGnRTi/3PV/w

    You can call OBJECT IDLETIME sess:n5Zq9HuyiSQJqXjiPDprHoxG.9OvYheqbq9Go1E13VMUQd7c+ZUA+5NXyGnRTi/3PV/w

    You will get a nil if that session no longer exists.
    OR you will get a number in seconds that it was last touched (written to) [ie. active at some level on your site].
    The seconds returned are bounded by 10 second increments, so it is pretty accurate. I am using this in part to figure out the ‘chance’ of that person still being actively online. i.e if a high int is returned they may be idle (grabbing a coffee) and not ‘actively’ online (although online).

  18. Joakim Kolsjö chimed in:

    I’ve made a gem inspired by this post: https://github.com/joakimk/session_tracker.

  19. John chimed in:

    There’s no need for a cron-job in order to get rid of “old” keys. Just get rid of all other keys, except the ones of the last 5 minutes while tracking.

    def track_user(id)
    redis.sadd current_key, id
    obsolete = redis.keys(“online_users.*”) – keys_in_last_5_minutes
    redis.del obsolete unless obsolete.empty?

    end

  20. anonymous chimed in:

    great solution. thank you!

  21. Dave chimed in:

    Brilliant post! Such a quick and elegant solution!

  22. Jason chimed in:

    You use the key online_users_minute_?
    after 1 hour pass, will it count old record?

  23. Wyatt chimed in:

    Just implemented this. Worked wonderfully. Thanks!

Leave a Reply

LukeMelia.com created 1999. ··· Luke Melia created 1976. ··· Live With Passion!