I use RSpec in allmyprojects. It’s really hard to overemphasize how helpful it is and how much easier becomes your life if you have good specs coverage. But its outstanding flexibility enables many ways to make your specs awful: horribly slow, over-bloated, even non-readable sometimes. I do not want to teach you BDD and RSpec here, but instead I will give you some ideas how to improve your specs quality and increase efficiency of your BDD workflow.

1. Use before :all block carefully

Sometimes it looks like a good idea to create a test data in before :all block. But be careful — these blocks are not wrapped in a transaction, so the data will not be rolled back after the test. In this case you should clear your data in the after :all block manually.

Another option is to move your before :all blocks to before :each to make them rolled back automatically.

2. For each test create exactly what it needs

Fixtures are cool when you start working on a project. But they quickly become painful while project grows: you add a new field to a fixture and break a half of your tests. There are tons of plugins which could simplify test data creation, I personally recommend factory_girl: it’s pretty slick and easy to use.

3. Do not create hundreds of records for a particular spec

Sometimes you want to test a method which operates on large set of records (filtering, trimming, etc). For example, this method returns 50 most popular videos (and no more). The straight approach is to create 51 record and make sure, that the size of the returned array is 50. When I saw a code snippet like this in our project first time, I was surprised. There was a few more pieces sharing this behavior, so here is my advice: add a parameter to the method, which will limit the number of records to return. In this case you can create 3 records, and pass 2 as a parameter.

4. Do not over-mock

Mocking is interesting and sometimes very useful technology. You may mock just everything so you spec will not hit the database. But there is a catch: your model code may be changed some day causing callers to break. Since you mock everything, you will never get failing specs. So now you should update all your mocks to fit a new interface. Also you would not be able to find SQL queries errors if you have mocked them. Instead of this I use integration approach: controller should talk to models, which have to hit the database. Real database with real data (OK, not so real). The practice 2 can help you in test data creation.

But you can use mocks to skip records retrieving from the database (make sure you have specs covering corresponding model code). Let me explain this. For example, you need to render 20 entries in an RSS feed. You could create 21 record in the database using a factory, and then ensure only 20 of them were retrieved, or you could mock your finder method and check its parameter. You may not like magic numbers like 20 in this particular case, and this is a good point. Just move this magic number to the config and ensure it was used to do the retrieval.

Usually you have more than one it block for each method. To group related specs I recommend to use nested describe blocks. Since describe is aliased to context when placed inside another describe, I think it’s a good idea to use it for specs grouping. Each context may have its own before and after blocks (in this case parent blocks will be called right before child ones).

6. Create several test suites to speed up your workflow

There are many things you can do to make your BDD more efficient. We will take a look at two of them: creating a separate test suites and running recently modified specs.

There are several standard test suites configured in RSpec by default: spec:controllers, spec:views, spec:helpers, spec:lib. Check the rake -T spec output to get a list of available RSpec tasks. Let’s create a simple Rake tasks generator for spec suites:

7. Stop spec_helper from being loaded multiple times

Just don’t do that. If you got a big project, there is a chance that the spec_helper will be required in a many different ways: File.expand_path, File.join, etc.,— which results in it being loaded several times and it slows down your test suite!

To avoid this, add the following code at the top of your spec_helper.rb:

12345678910111213141516171819202122

# figure out where we are being loaded fromif$LOADED_FEATURES.grep(/spec\/spec_helper\.rb/).any?beginraise"foo"rescue=> eputs<<-MSG
===================================================
It looks like spec_helper.rb has been loaded
multiple times. Normalize the require to:

require"spec/spec_helper"

Things like File.joinandFile.expand_path will
cause it to be loaded multiple times.

It will show where you’ve tried to load spec_helper from so you will be able to fix it immediately. Also there is an interesting snippet of code here, which will find and replace all wrong includes.

Conclusion

RSpec is not a silver bullet. You can have 100% coverage and fine-grained specs, but it does not mean your application is completely bug-free. Refactor your specs, increase your programming level, and refactor again. Write specs for any issue that you, your QAs or users have faced. And remember: do not over-mock.

Credits

This is my first article written completely in Google Wave in collaboration with several good Russian rubyists. Thank you all, guys. I want to acknowledge the editorial help of Roman Dmytrenko and Alexey Kovyrin. Robby Russell created a great picture illustrating how sexy is RSpec. And thank you all my readers for your attention.

27 Responses to this entry

Nice practices, thanks for an interesting article. I think that’s a kind of knowledge that Rails community needs to share, as we have too much practices that are very dogmatic and unrealistic.

Unfortunately, RSpec has a documentation that tries to enforce mocking and stubbing for all controller specs. Not only it could bloat specs code with mocks/stubs, but also create a sense of false security, so you think your code works when it’s not.

I also consider over-mocking a big evil, especially when it comes to chaining several scopes, or some branching logic in controller, or… well, it messes up with a lot of use cases. I learned that hard way! Ideally I would recommend to use mocks in the following situations:

Stubbing out the external service interaction, unless it’s the whole point of a given test;

Skipping complex activity irrelevant to a given test (like post-processing some data in :after_create block, which is already tested on its own);

Defaulting some complex queries to a manually crafted data (similar to what you described with top videos).

There are always exceptions though. But the good rule of a thumb would be to rely on spec robustness: how likely is that the spec fails when the process it tests no longer works for some reasons? Do mocks help to fail earlier, or delay the time required to react to application errors?

Those are good points. I did have some specs so far mocked out that the didn’t produce any relevant failures. Usually I try to cover that with cucumber features.
But I will try your approach on my next controller and see what it leads to.

If you rather use the spec command, you’d be thrilled to know that it actually defaults to fetch the options from spec/spec.opts (and, of course, you could always explicitely aliasspec='spec -O spec/spec.opts'). :)

Well, there must be a way to configure the text editors and IDEs (I just checked rspec-tmbundle’s Spec::Mate::Runner and it takes arbitrary options everywhere it makes sense to do so, and given that it’s RSpec’s David Chelimsky’s code, I’d be surprised if it didn’t handle the spec/spec.opts case sanely or even defaulted to it).

I don’t really get the ‘there is too much work to exclude this single line from specs’ comment; I simply never put any such line in my specs to begin with (I hate clutter and never understood the popular custom of starting each and every spec with the same noise line, when the same behaviour can be configured in one place).

But I definitely don’t want to evangelise; I just wanted to share my (IMHO, saner and more elegant) solution. Thanks for all the other great tips! :)

rake spec and spec spec return different results on my Rails app. Why are they different and how to determine when to use which command. From my observation, one use data from development db and the other use data from fixture.

The only difference is that rake spec executes db:test:prepare before running your test suite. It means, that spec spec is faster (on large databases — significantly faster), but you will have to run rake db:migrate RAILS_ENV=test manually to ensure your test database is up to day.

I’ve tried using Rspec before and find that it takes longer to code the tests than to write the code correct in the first place. Maybe I’ve been doing this far too long (35 years now). Perhaps this system is useful for teams of people working on a project but for a single developer, I really don’t see the advantage of spending the time doing this when the time could be spent more productively.

Indeed, it is faster to write the code in the first place. In some languages, such as Lisp (or Scheme), incredibly so, using the REPL. And, in the case of functional languages (NOT ruby), you can even prove that the code is correct.

The problem with that is when things change. And they *will*. You can’t do simple refactorings like changing a method name, while knowing for sure that your change didn’t break an obscure section in your app. Or, when you figure out a more efficient way to do something, it your business logic is still functioning.

Okay. Please, share your experience with us. What makes you sure that your code working and working right? How easy will it be for you to change some logic in your project or just do a refactoring and be sure that you didn’t broke something your project?

These are some good tips, but I don’t fully agree with all them. Particular points 4 and 5:

About mocks: In general, I agree with you that over mocking is a problem. But I disagree that it’s always better to use no mocking or stubbing in your controller specs. In my opinion, the danger of mocking is directly proportional to how likely it is the interface you are mocking will change. So, mocking a class method that frequently changes its interface is dangerous and should be avoided. Mocking an interface that is highly unlikely to change is generally safe and can lead to less brittle tests.

So, for example, when I’m spec’ing the #update and #create actions of a controller, I mock #valid? so that it returns true or false based on which code path I want to test. The alternative is to pass valid or invalid attributes in the request, but this leads to fragile specs: now my controller specs have to know what attribute values are valid and invalid, and when my model validation logic changes, my controller specs begin to fail. Really, I don’t care in my controller specs what is valid and invalid…I just care that when the attributes are valid, it does one thing, and when they are invalid, it does another thing. #valid? has been around in ActiveRecord for as long as I can remember (I imagine it was there from the first release) and the likely hood of that interface ever changing is approximately nil. I think it’s safe and useful to use mocking here.

About describe vs. context: In your example, you use context for both “.top” and “when just created”. I think context is appropriate for the latter (since “when just created” describes a context the examples run in) but describe is more appropriate for the former. “.top” isn’t a context; rather it’s simply grouping all the examples where you are describing the top method. I use use describe in these cases since you’re describing the behavior of a method.

Hey Myron, thank you for the reply. I have updated an article with more consistent context usage, it was my failure.

About mocking in controllers, yes it makes sense. Sometimes. When you pretty sure you models work as expected, when you confident in their work. As I said, in some circumstances you can get false positives, for example, when add after_create hook which fails with an exception, or new super-useful NOTNULL restriction on one of columns in the database. If you have all your models covered with — it’s ok, use mocks to get controller specs clean in simple. By the way, did you hear about stub_model?

Second, the problem of missing problems is only a problem if you don’t use integration tests and stories.

You have classes A and B. Each should be tested in near-isolation. You especially want to isolate from the database, for various reasons. BUT… you also need to have an integration test that runs A->B->database. If the later breaks, fixing it will break the obsolete unit tests, which is as it should be.

The other thing you did not seem to mention is the importance of testing interfaces, not implementations. Maybe that is a no-brainer to you, but code changes much faster than the interface.

# figure out where we are being loaded from# see: http://kpumuk.info/ruby-on-rails/my-top-7-rspec-best-practices/def prevent_multiple_require(regex, standardized_require_path)if$LOADED_FEATURES.grep(regex).any?beginraise"foo"rescue=> eputs<<-MSG
===================================================
It looks like #{File.basename(standardized_require_path)}.rb has been loaded
multiple times. Normalize the require to:

require"#{standardized_require_path}"

Things like File.joinandFile.expand_path will
cause it to be loaded multiple times.

This improves on #7 because you can customize the regex used for matching, and reuse it for other situations. I have made it as simple as possible because other paths that may require spec_helper may use ‘..’.

There is no need to raise an exception in order to get a backtrace; Kernel#caller provides the desired behaviour. Also, it is good practice to use Kernel#warn for warnings (which sends output to $stderr by default), instead of puts-ing out to $stdout where it is inseparable from the output of your specs:

12345678910111213141516171819

# figure out where we are being loaded fromif$LOADED_FEATURES.grep(/spec\/spec_helper\.rb$/).any?
warn <<-MSG
===================================================
It looks like spec_helper.rb has been loaded
multiple times. Normalize the require to:

require"spec/spec_helper"

Things like File.joinandFile.expand_path will
cause it to be loaded multiple times.

Post a comment

Name (required)

Email (will not be published) (required)

Website

You can use simple HTML-formatting tags (like <a>, <strong>, <em>, <ul>, <blockquote>, and other). To format your code sample use [cclang="php"]$a = "hello";[/cc] (allowed languages are ruby, php, yaml, html, csharp, javascript). Also you can use [cc][/cc] block and its syntax would not be highlighted.