README.markdown

Riot

Riot Rails: Rails support for Riot testing. A definite work in progress (help welcomed). See evoke-app for live examples.

Riot.js: for you JavaScript people, a version of Riot just for you. Both implementations will be informing the other. Lots of stuff to learn.

Installation

The Riot gem is hosted on gemcutter.org. It used to be hosted on GitHub, but they turned off gem support while moving to Rackspace, so we moved to gemcutter. If you have not already done so, add gemcutter.org to your list of gem sources like so:

sudo gem sources -a http://gemcutter.org

Then, simply install the Riot gem like so:

sudo gem install riot

Note on speed

I have done a really simple benchmarking (10,000 runs), but right now, Riot is running about 2 times faster than Test::unit and thusly Shoulda:

Riot also works very well with Ruby 1.9. The same benchmarks from above run through ruby-1.9 show Riot to be twice as fast as it is already. See our benchmarks gist for more details.

Examples

Example: Basic booleans

NOTE: For no specific reason, I'm going to use an ActiveRecord model in the following examples.

At it's very basic, Riot simply tries to assert that an expression is true or false. Riot does this through its asserts or should tests. An asserts test will pass if the result of running the test is neither nil or false. A should test does the exact same thing - they are in effect, aliases. The only difference is in the way you write the assertion.

For instance, given a test file named foo_test.rb, you might have the following code in it:

Sometimes it's more clear to say "this should be that" and sometimes it's better to say "asserts this is that". I promise you that Riot will get no more redundant than this, but also that besides speed, Riot will aim at being expressive with a minimal amount of syntax.

The other important thing to note in the examples above is the use of the topic. Calling topic within any assertion will actually return the value of whatever was evaluated and returned from calling the last setup in the given context. In the examples above, User.new was returned, and is therefor accessible as the topic.

I'm going to use asserts for the rest of this introduction, but you should know that you can replace any instance of asserts with should and nothing would change.

Example: Shortcut - Asserting the topic itself

Over the course of developing Riot it became somewhat obvious to some of us that we were creating assertions that returned the topic just so we could assert things about the topic itself. For instance, were doing this:

If you'd like to add a description to your assertion about the topic,
but still would like to avoid the redundancy of an assertion simply
returning the topic from it's block, you can omit the block all
together as follows:

Example: Equality

One of the most common assertions you will (or do already) utilize is that of equality; is this equal to that? Riot supports this in a slightly different manner than most other frameworks. With Riot, you add the expectation to the assertion itself.

Here, you should begin to notice that tests themselves return the actual value. You do not write assertions into the test. Assertions are "aspected" onto the test. If the test above did not return 'foo@bar.com' for topic.email, the assertion would have failed.

The equals modifier works with any type of value, including nil's. However, if you wanted to test for nil explicitly, you could simply do this:

Example: Nested contexts

Oh yeah, Riot does those, too. The setup from each parent is "loaded" into the context and then the context is executed as normal. Test naming is a composite of the parent contexts' names. Here, we'll do a little Sinatra testing (see below for instructions on how to make it Riot work seamlessly with Sinatra tests).

You say, "OMG! Why did you write this?"

Some background, I guess

You start a new project. You get all excited. You're adding tests. You're adding factories. You're fixturating your setups. You're adding more tests. Your tests start slowing down, but you need to keep pushing because your backlog has a lot of new, nifty features in it. You've got 3000+ lines of test code, 2000+ assertions. Your tests are annoyingly slow. Your tests have become a burden.

I hate this and it happens a lot, even when I'm conscience that it's happening. Even when I'm not hitting the database and I'm mocking the crap out my code.

I really, really hate slow test suites.

Did ... you look at Shoulda

I should say that I love Shoulda in theory and in practice. It changed the way I coded. I added macros all over the place. I built macros into the gems I wrote for the gem itself. But, alas, Shoulda is slow. Shoulda is based on Test::Unit. Shoulda reruns setups for every should. Shoulda could make my life even easier with even more expressiveness.

Did ... you look at RSpec

:| yes, no, I don't care. It's slow, too. Anyways, I was at ObjectMentor many, many moons ago when Dave Astels (accompanied by David Chelimsky) were explaining this brand new approach to testing called BDD. Mr. Astels explained to us that we if we already understood TDD, then BDD wouldn't help a bunch. Why argue with him?

How Riot is the same

It defines a context

It prints .'s, F's, and E's when tests pass, fail, or error

It tells you how long it took to run just the tests

Contexts can be nested and setups inherited

How Riot is different

Riot differs primarily in that it does not rerun setup for each test in a context. I know this is going to shock a lot of folks. However, over the past several years of my doing TDD in some capacity or another, there are certain habits I have tried to pick up on any many others I have tried to drop.

For instance, I believe that no assertion should mangle the context of the test data it is running in. Following this allows me to require setup be run only once for a collection of related assertions. Even in a nested context where setups are inherited, the setup's are called only once per the specific context.

Following all of this allows me to have very fast tests (so far).

...

Riot is also different in that assertions are not added to the test block. Each test block is it's own assertion (and assertion block). Whatever the result is of processing an assertion block will be used to determine if an assertion passed or failed. Each assertion block can have a specific validator tied to it for asserting any number of things, like: the result of the test equals some expected value, the result of the test matches some expected expression, or the result of the test raises some exception.

I like this approach because I only want to test one thing, but that one thing may require some work on my behalf to get the value. Riot does not let me cheat in this regard. There is no way for me to add more than one assertion to an assertion block.

...

I imagine this approach will persuade many of you to avoid Riot altogether. I don't blame you. A few years ago I would have avoided it, too. As of this writing though, I have ported Chicago and Slvu (which were previously written in Test::Unit + Shoulda) to Riot, cut the number of lines of code in almost half, definitely more than halved the amount of time the tests took to run, and did so in less than half a day (I was building Riot while porting them :).

Running tests

Create or modify your existing Rakefile to define a test task like so:

Basically, it's like setting up any other test runner. Then, from the command line, you only need to run rake or rake test. Please make sure to remove all references to any other testing frameworks before running tests. For instance, do not require test/unit, shoulda, minitest, or anything else like it.

With Sinatra

Riot definitely works with the latest Sinatra. I personally use it to run tests for Chicago and Slvu. Setup is pretty easy and very much like getting your tests to run with Test::Unit. In a test helper file that gets loaded into all of your tests (that need it), enter the following:

With Rails

It's doubtful that Riot works with Rails very easily as Riot completely replaces Test::Unit. I haven't tried it yet, but intend to with my next new Rails project. Porting would probably take some time unless you only have a few test cases. Porting is made somewhat easier if you're already using Shoulda; you can replace the TestCase definition with a context of the same name as the class under test I suppose.

Extending Riot with Macros

To extend Riot, similar to how you would with Shoulda, you simply need to include your methods into the Riot::Context class. For example, let's say you wanted to add a helpful macro for asserting the response status of some HTTP result in Sinatra. You could do this easily by defining your macro like so:

Assertion macros

If you want to add special macros to an Assertion, this is as easy as adding them to a Context. Assertion macros, however, have a special mechanism for adding themselves onto an assertion. Thus, you will want to open the Riot::Assertion class and then define your assertion macro.

For instance, let's say you wanted to add a macro for verifying that the result of an assertion is the same kind_of object as you would expect. You would define the macro like so:

If you think you might want to chain assertions checks together, this won't work out the box. The reason for this is that your assertion macro should call out to pass or fail, which each return a signal that will be used by the reporter. You could conceivably write an assertion as a filter macro instead, that would return self, but I can't decide why you would do that yet (in theory it would work, though).