Performance on Rails

Solution 1: Eager Fetching with :include

Rails provides an easy way to solve basic n+1 issues: eager fetching with the :include option. When fetching the post, we can fetch all of its comments and each corresponding user in far fewer SQL queries:

In versions of Rails prior to 2.1, this statement would fetch the post, its comments, and the users with a single complicated query composed of SQL JOINs. The current version (2.1 at this writing) does not use JOINs, instead favoring one query per model that you specify in your :include parameter:

In this example, one query fetches the post, a second query fetches the comments for that post, and a third query fetches the users that wrote the comments. This approach makes for a couple of extra queries, but JOINs can be expensive and become a bottleneck in their own right. Either approach is a big improvement over the original n+1 problem.

Solution 2: Eager Fetching with JOIN

There are times that the :include option alone isn't sufficient. What happens if a post is wildly popular and has comments by hundreds, even thousands, of users? The IN clause in the users query will have a lot of IDs, and performance will begin to sufferthe query may even fail if the list of IDs is too long. The solution is to fetch even more eagerly using a JOIN in your query. Instead of writing a query for find_by_sql, encourage Active Record to use a JOIN for you:

include => :user, :conditions => 'users.id is not null'

When you include a filter on users.id in the :conditions option, Active Record smartly fetches the users with a JOIN (specifically a LEFT OUTER JOIN) to satisfy the dependency you've introduced in the WHERE clause on the users table. It's probably a good idea to comment code like this to document the intent of the :conditions option.

Solution 3: has_many :through

Posts have a transitive dependency on users: A post has comments, and each comment has a user; therefore, a post has commenters. Use Active Record's has_many :through to declare this dependency:

Note that I'm calling #detect, not #find. This is because I want to invoke the method provided by the Enumerable module, and ActiveRecord::Base#find overrides Enumerable#find.

This strategy works pretty well for a small number of commenters; however, #detect performs a sequential search, an O(n) operation. This method won't perform well for a large value of n, when there are a lot of commenters. You might want to build a lookup hash for constant-time lookup, or O(1):

Solution 4: Aggregate

Another example of when the :include option isn't enough is when you are fetching aggregated data. Let's say you'd like to fetch the number of comments each user has created to display that next to the user's name. If you calculate it user by user (via comment.user.comments.count), you'll have another n+1 problem. One approach is to calculate the data for all relevant users in a single SQL query and build a lookup hash.

This code, which could be in a controller or model, gets the number of comments for each user by grouping by user_id and counting the rows in each group with the SQL COUNT function. The #group_by method creates a hash where the key is the user_id (because it's returned by the block), and the value is an array containing all items with that key. The view can use this hash to look up the count for a given user:

The view looks up the record for the comment's user_id in the hash, and needs to call #first to pull the one and only record out of the array before getting the count.

Code like this should be well commented, particularly the fact that it addresses a performance issue. There is another option for this kind of problem: denormalization.

Solution 5: Denormalize

"Normalize until it hurts. Denormalize until it works."

In a perfectly normalized database, there is only one representation of any particular fact. Taken to the extreme, this results in a space-efficient database with no chance of duplication or inconsistency. This is a wonderful ideal, but it comes at a cost: time-efficiency. We software developers walk a fine line between idealism and pragmatism.

Active Record makes it very easy to denormalize the number of comments that a user has created. Just enable a counter cache on the association:

And create a migration to add an integer column named comments_count to the users table. Here's a migration that adds the column and calculates the count for each user because our users have created comments before we added this denormalization:

Every time users create a new comment, their comments counter cache is incremented, and if they delete a comment, the counter cache is decremented. Now the view can display the user's comment count without any additional database queries:

<%= comment.user.comments_count %>

Counter caches are a simple type of denormalization that is built into Active Record, but your requirements might be more complex. Let's say you want to denormalize the date and time of the user's first comment. I would use the before_create and before_destroy lifecycle hooks to keep the data in sync:

Conclusion

Ruby the language and Rails the framework often take a beating from detractors on the question of performance. Ruby's not in the running to be the fastest language, and it's not even the fastest interpreted language. Rails isn't the fastest framework. However, if Ruby or Rails is your bottleneck, consider yourself lucky! Most unexpected performance challenges are related to querying the database and aren't detectable until the app has been in the wild, which is no different than any other software development framework.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!