Testing Active Record Scopes

Active Record scopes are an interesting thing to test. In projects I’ve worked on, I have seen many different patterns of testing, some much better than others.

A little over two years ago I wrote the gem pg_search, which provides a sort of Domain-Specific Language (DSL) for creating Active Record scopes that take advantage of PostgreSQL’s built-in full-text search.

For example, the following code will set up a scope that accepts query as a string and returns records whose name matches that query. The records will be ordered by relevance.

In pg_search’s test suite, I found myself needing to test the results of scopes over and over again. In doing so, I believe I have developed a reasonable approach. But first, let’s look at a common pitfall.

In this example, we use an RSpec test double to simulate what would happen if we were to call Book.order("year ASC"). We then call our Book.chronological method to see if it returns the same value.

There are a few problems with this approach. First off, we are testing a class method on the Book class, and also stubbing the class method Book.order. By modifying part of how Book works, we cannot be sure that the object will work when the mocks are absent.

Secondly, our test code is tightly coupled to our implementation. What if we decide that the .chronological scope should always rank records with NULL year first? In PostgreSQL, this would work:

def self.chronological
order("year ASC NULLS FIRST")
end

This is effectively a new feature, but you have to go back and change the test for an old feature in order to make everything pass.

In this example, we set up a much simpler situation. We create three records, then expect the chronological method to return them in order. Note that we use Enumerable#index to compare the positions of the records in the output Array.

All of these code examples should pass the test. And our test no longer cares about that joins(:author) part. The original stubbed version would have failed because .order is not called directly on Book anymore.

Avoiding brittleness

We could also have written something like this:

results.should == [earlier, middle, later]

But now that the database is involved, we would need to be more careful to allow for other records that might be there, such as test fixtures. We could solve that by deleting all Book records at the beginning of the spec.

My personal preference is to avoid the Book.delete_all solution. In my opinion it’s better to have a test that works regardless of the internal state of other objects and systems.

Also, if later down the road a test fixture happens to be present and somehow breaks my test, I have a chance of noticing and doing something about it at that point. If I always delete all of the records, then this free informal fuzz testing goes away.

And any class method you define on Book is callable directly on the Relation instance. It’s almost as if it were a subclass of Book. Even silly methods that have nothing to do with the database work almost as normal.

And yet Relation objects also act like an Enumerable and have all the methods like #each, #map, #select, and so on. In fact, there is the crazy implementation of ActiveRecord::QueryMethods#select which goes out of its way to guess whether you wanted the Active Record class method or the Enumerable method.

In closing

So scopes are these strange methods that have a fluent interface that chains, create a set of virutal subclasses of your model, and build an abstract syntax tree (AST) of your SQL queries using the Arel gem.

This last point is the nail in the coffin for me. An AST is not that meaningful until it is compiled into runnable code and executed. And with scopes and Relation objects, that means generating SQL code. And SQL code itself is not that interesting until it is executed against a database. For most of its useful operations, a Relation is actually a code generator for another language!

So my vote is to embrace the database, get something working quickly, and move on. That way, you know that your true goals are going to work against your actual application.

I’d love to hear feedback and debate. I don’t believe this is a closed issue. Feel free to join the discussion in the comments!

2 Comments

Robbie Clutton says:

Excellent. I think testing scopes is essential but also not a unit test. It is an integration because of the abstraction from ActiveRecord to Database driver SQL to database and back again. I wrote up some strategies where you can mix tests that hit the DB and those that don’t within a model test here: http://pivotallabs.com/testing-strategies-rspec-nulldb-nosql/