The End Result

Weplay members were told us they wanted to be able to see which of their friends were online, so we decided to add the feature. Let’s look at how Redis was instrumental in this functionality.

Properties of a Set

First a quick overview of Redis’s “set” data structure. A set in Redis has the same properties as the abstract data structure of the same name:

* 0 to N elements
* Unordered
* No repeated members

In practice, this is nice because adding a value to a set does not require you to check if the value exists in the set first.

Redis Set Operations

You can see a full list of the set operations Redis supports. here are the ones were going to use:

The Approach

The idea is to have one active set per minute. During each request that comes in from a logged-in user, we’ll add a user ID to the active set. When we want to know which user IDs are online, we can union the last 5 sets to get a collection of user IDs who have made a request in the last 5 minutes.

Now, if we have a set of the user’s friend’s IDs, we can intersect that with the online users and we’ve got our online friend IDs.

The Code

Here’s the relevant code. (Note: we’re using the redis-rb gem, by Ezra Zygmuntowicz (aka @ezmobius), as our Redis client.)

I love that the code is almost as succinct as the diagrams.

With this approach, you need to empty or delete the older sets before the next hour comes around. We use a cron job for that purpose. I’m not going to show that here, but feel free to get in touch if you have any questions about it.

Hope this gives you some ideas of the Redis’ elegant simplicity and power. This code simpler than it would be if we had used an RDBMS or memcached, or both.

If you’re new to Redis, I’d recommend following it’s creator and maintainer, Salvatore Sanfilippo (@antirez). He’s a very thoughtful and transparent open source leader and is rapidly and continually improving Redis.

As of now (October, 2009), EngineYard doesn’t support Ruby Enterprise Edition, so we had to go it alone. This blog post documents our configuration.

We run in EngineYard’s cluster environment with nginx -> HAProxy -> mongrel load-balanced across a few app server slices with a utility slice running delayed_job workers. Our ruby processes are managed by monit.

Here’s how we went about things:

Install REE

I installed REE at /opt/ruby-enterprise-1.8.6-20090610. One of my goals was a side-by-side installation with MRI so that I could switch back quickly if something went wrong. (As it’s turned out, REE has been rock-solid for us.)

Note the –auto flag which provides for a non-interactive installation.

Update mongrel_rails script

Our EngineYard slices have MRI installed at /usr/bin/ruby18 with a symlink at /usr/bin/ruby. The shebang line in /usr/bin/mongrel_rails was:

#!/usr/bin/ruby18

Change it to:

#!/usr/bin/ruby

This may not be absolutely necessary based on how your environment starts mongrel, but I’d
recommend doing it anyway.

Prepare a script to update symlinks

So now you’ll have the following setup:

/opt/ruby-enterprise-1.8.6-20090610/bin/ruby

/usr/bin/ruby18

/usr/bin/ruby -> ruby18

To switch to REE, we’re going to switch that symlink to point at the REE ruby binary. You’ll also want to do the same for other binaries like rake, gem, etc. Here’s a script, eyruby_switch.rb, I wrote to switch them all at once:

You will call the script like this: ./eyruby_switch.rb ree or ./eyruby_switch.rb mri. Don’t execute it just yet.

Re-install gems

Before the switch, I wanted to get all our non-vendored gems installed in REE. The REE install has its own gems directory, and you can’t share gems with native extensions between REE and MRI anyway, so you need to reinstall. Here’s an example of installing a gem for REE without needing to have the symlinks pointed to REE:

sudo /opt/ruby-enterprise-1.8.6-20090610/bin/ruby /opt/ruby-enterprise-1.8.6-20090610/bin/gem install --no-rdoc --no-ri nokogiri

Setup GC-tuning wrapper script

For us, the big performance win for REE was the tuneable GC parameters. I’m not going to try to explain the params here. If you made it this far, you’ve probably already ready Evan Weaver’s blog posts on this topic.

Our goal was to apply the GC params to all rails processes, but not necessarily to every ruby process running on the system. So we created a wrapper script called ruby_with_env:

Put it somewhere on your system (ours is at /data/weplay/shared/bin/ruby_with_env) and make sure it’s executable.

Update monit scripts

Examine all the scripts specified by the .monitrc files in /etc/monit.d/. In any script that starts a rails process, replace the call to /usr/bin/ruby with the ruby_with_env wrapper script. This is safe to do even if MRI is still the active ruby on the system, because the environment variables will simply have no effect with MRI — everything else should run properly.

Flip the switch, hero

Update the symlinks:

./eyruby_switch.rb ree

Verify the script worked with ruby -v

Then tell monit to restart your processes and your app is now running on REE.

Feedback

I would love to see EngineYard support REE so I can use their otherwise excellent team to do setup and maintain this stuff for me. Until then, maybe we can help each other out. If you have questions or ideas to improve this configuration, let me know in the comments or on twitter.

One more thing

If you’re in the early stages of exploring REE and you just want to try it out locally, I highly recommend Wayne Seguin’s awesome rvm – Ruby Version Manager. It makes it easy to install REE and switch back in forth in a development environment.

One thing I’ve learned about myself is that I get more out of a conference if I a) am well rested, and b) take notes. With 7.5 hours of sleep on Friday night (a lot for me), I fired up TextMate during the opening talk and started the note-taking. I made an offhand comment in the #nyc.rb IRC room that collaborative note taking would be cool. Ari Brown suggested EtherPad, which turned out to be a great tool for the job.

EtherPad is pretty impressive. It allows multiple authors to simultaneously edit a document. You can see your own and other people’s edits and contributions highlighted in a color associated with each contributor. It supports simple-formatting, a little chat window, saved revisions, and nice import/export options. And it does this all in a web browser without plugins, logins, or any friction whatsoever. Big props to the development team behind that thing.

The primary note takers were Ari and me, but one particularly cool moment in the collaborative experience was when Gregory Brown, the lead-off speaker, came on and did a light edit of the notes from his own talk.

Link: Notes from GoRuCo

But I haven’t been enjoying supporting both Prototype/script.aculo.us and jQuery on in the same web app. It felt downright wrong to make my visitors download all that javascript.

So, this weekend, I set out to eliminate the last of the public-facing prototype-dependent code from weplay and gained some peace of mind along with a 50% reduction in javascript bytes required for the fattest page.

Here are a few tips based on my experience so far:

1. Grab your inheritance. For porting components that rely on Prototype’s Class.create mechanism, the most straightforward path is to leverage a jQuery-native implementation of the same concept. I used Dan Webb’s $.klass code from his jQuery version of LowPro. I also came across the jquery-klass project, which looks like it would do the job.

   It feels a little dirty to bring Prototype’s inheritance-based OO approach into jQuery — the jQuery philosophy is more about embracing the prototype-based OO baked into the language. As part of a porting effort, though, I think this approach makes sense, at least for a step.

2. Firebug is your friend. After I addressed the Class.create challenge, my approach was basically to repeatedly load the page in the Firefox and fix the first error that Firebug reported. By the time I worked through them all, dealing with a few remaining IE-specific issues was not too bad.
3. Enjoy watching the code shrink. Here’s a bit of code before and after the port…

   Before:

   this._arrow = document.createElement('img');
   this._arrow.border = 0;
   this._arrow.className = 'color_picker_arrow';
   this._arrow.src = this.settings.arrowImage;
   this._arrow.margin = 0;
   this._arrow.padding = 0;
   this._arrow.style.position = 'absolute';
   this._arrow.style.top = '0px';
   this._arrow.style.left = '0px';
   this._arrow.style.zIndex = 10000;
   document.body.appendChild(this._arrow);
   

   After:

   this._arrow = jQuery(document.createElement('img'))
   .attr({border: 0, src: this.settings.arrowImage})
   .addClass('color_picker_arrow')
   .css({margin:0, padding:0, position:'absolute', top:0, left:0, zIndex:10000})
   .appendTo(document.body);
   

4. Grok the jQuery event model. jQuery has excellent support for working with events, but it is different from Prototype’s. In particular, Prototype’s bindAsEventListener method, which changes what this refers to in your event handler, is nowhere to be found. Figure out how to use closure to attain the same thing and revel in jQuery’s simpler approach to “this”.
5. Consider jRails. If you’re working in Rails and using the link_to_remote methods, consider jRails as a stop-gap measure to help you show Prototype the door. I say “stop-gap” because link_to_remote does not produce unobtrusive javascript and does not tend to support real progressive enhancement.
6. Bonus Tip: Learn to write jQuery plugins. While not strictly required by my porting efforts, I wrote a few plugins along the way to help out. Writing a plugin sounds like it could be a big deal, but writing a jQuery plugin is very straightforward, and it’s an addictive way to package up functionality as well as get more comfortable with the jQuery philosophy.

Good luck, and share your porting stories!

UPDATE: A few more conference pics, courtesy of Sebastian, including some of the afterparty/hackfest, where Jeanhee & Chiara came out to meet the ruby geeks!

Install it with sake from here.

UPDATE: If you’re interested in watching Ryan’s stellar talk, it will be up on Confreaks soon, or you can try the RubyConf edition of the talk.

Anyway, lots of great changes since 1.043. Check it out and let us know what you think.