Getting Testy: Learning

Introduction

In the
previous post,
we talked about writing good acceptance or story tests. Now that
that outer layer of tests is in place, it’s time to start developing
the application itself.

As you may recall, we’re building a simple application that can do two
things:

Convert an amount of money from one currency to another.

Show a list of all currencies supported by the application.

We still haven’t decided what kind of application this should be, but
we’re at the point where we need to make that decision.

Let’s start with a command-line application just to get things
running. We may convert it into a web application later, or maybe
just add a web front-end and allow it to be used either way.

What Now?

With any new project, there are likely some things we’re not sure
about. For our command-line application, we might need to learn about
how to handle the command-line options. For a web application, we
might want to try out Sinatra but aren’t quite sure how it works. We
also might want to experiment with a few different APIs for getting at
the currency data we need for the application.

One way to resolve these uncertainties is to open up
irb and
start poking around. But another way is to write learning tests, an
idea that Kent Beck credits to Jim Newkirk and Laurent Bossavit in his
Test-Driven Development By Example
book. James Grenning has also written
a nice blog post
on the subject.

We can use our test framework to help us explore the landscape and to
capture what we learn along the way.

Learning OptionParser

Let’s look at how we might apply learning tests to Ruby’s
OptionParser. We can first review
the documentation
to get a general sense of how to use the class, but we’ll really want
to experiment with a few ideas.

Simple Help Option

Let’s figure out how to get a basic help option to work with
OptionParser. We’ll use Minitest for this post.

From reading the documentation, it looks like we need to implement the
help option if we want one. The implementation should print some kind
of usage string on $stdout and then exit. We can use Minitest’s
assert_output for that.

There’s a good example of how to implement the help option in the
documentation, but let’s first make sure we can create a basic
OptionParser and see what it does. We don’t know the exact format
of the output, so we can just expect an empty string and the assertion
failure will tell us.

All of the examples in the documentation use the parse! method, and
it looks like we can pass in an array of strings in order to simulate
the command-line arguments passed to the program. That seems really
handy for tests, so let’s do that.

Test for Basic Help Command

classOptionsTest<Minitest::Test

deftest_help

parser=OptionParser.new

assert_output("","")do

parser.parse!(%w[-h])

end

end

end

assert_output takes two positional parameters and a block. The
block is run, and any output to $stdout and $stderr is captured
and compared against the two parameters. If the output matches, the
test passes; if not, it fails.

When we run these tests, they just abort prematurely. After a little
bit of digging, we figure out that OptionParser is exiting for us.
That’s not too helpful when you want to run an entire test suite.

A bit of searching tells us that we can stop the exit from happening
by rescuing the SystemExit exception. Minitest has assert_raises
that helps with this:

Don't Exit the Tests

deftest_help

parser=OptionParser.new

assert_output("","")do

assert_raises(SystemExit)do

parser.parse!(%w[-h])

end

end

end

Now we get a test failure, because the output doesn’t match the empty
string we passed in. Given that we haven’t implemented a help option
yet, we expected an error message to that effect. Instead, we got
actual output showing a usage message. OptionParser must have a
pre-built implementation of the help option, though the documentation
doesn’t really mention that.

The output contains a program name we don’t recognize. When using
Rake’s test task, the program name is rake_test_loader; when running
from RubyMine, it’s tunit_or_minitest_in_folder_runner. We don’t
really want to couple our tests to one way of running them, so we go
back to the documentation. It turns out that we can tell the
OptionParser what program name to use.

Supplying a Program Name

deftest_help

parser=OptionParser.newdo|p|

p.program_name="PROGRAM"

end

assert_output("Usage: PROGRAM [options]\n","")do

assert_raises(SystemExit)do

parser.parse!(%w[-h])

end

end

end

I’m using a simple, obvious string (PROGRAM) as the program name so
that it shows up clearly in the assertion failures.

Having a fixed program name allows us to fill in the proper expected
output string, and our first learning test is now passing.

Version Option

Every command-line application should be able to report its version.
We’ll write that test next. First, let’s pull the OptionParser up
into a setup method so we don’t have to keep redefining it in every
test.

Extracting a setup method

classOptionsTest<Minitest::Test

defsetup

super

@parser=OptionParser.newdo|parser|

parser.program_name="PROGRAM"

end

end

deftest_help

assert_output("Usage: PROGRAM [options]\n","")do

assert_raises(SystemExit)do

parser.parse!(%w[-h])

end

end

end

private

attr_reader:parser

end

I’ve used a private attr_reader for the parser instance variable
so that I can use
barewords. It
doesn’t strictly need to be private, but I’ve gotten in that habit to
keep from exposing internal state from my classes. Admittedly this is
less important in a test class.

Now we can write a test for the version option.

Test for Version Option

deftest_version

assert_output("","")do

parser.parse!(%w[--version])

end

end

Once again, we just pass in empty strings until we know the exact
output format. And once again, the tests exit prematurely.
OptionParser must be exiting for us here, too. Maybe it also
implements the version option for us? Let’s use the same trick we did
for the help option.

Trapping the Exit Again

deftest_version

assert_output("","")do

assert_raises(SystemExit)do

parser.parse!(%w[--version])

end

end

end

Once again, we get the test failure we expect. And, as we suspected,
there seems to be a built-in implementation of the version option, but
it’s saying version unknown. How do we tell it what version to
report? According to the documentation, there’s a version attribute
we can set.

Specifying the Version

defsetup

super

@parser=OptionParser.newdo|parser|

parser.program_name="PROGRAM"

parser.version="VERSION"

end

end

deftest_version

assert_output("PROGRAM VERSION\n","")do

assert_raises(SystemExit)do

parser.parse!(%w[--version])

end

end

end

Again, we use an obvious literal string for the version and add the
actual expected output. We have another passing test.

A Flag Option

Next, we’re curious about how to pass in a flag to enable an option.
Our application will need a way of knowing when to output a list of
currencies. One way we might do that is to allow a -l or --list
option.

From the examples in the documentation, it seems like the thing to do
is to define a Hash, Struct, or OpenStruct to hold the options,
and then have the OptionParser populate the options based on the
command-line flags. Let’s try that.

Test for List Flag Option

defsetup

super

@options=OpenStruct.new

@parser=OptionParser.newdo|parser|

parser.program_name="PROGRAM"

parser.version="VERSION"

parser.on("--list","Show currency list")do|flag|

options.show_list=flag

end

end

end

deftest_list_flag

parser.parse!(%w[--list])

assert(options.show_list)

end

attr_reader:options,:parser

The new test passes as soon as we add the list option to the parser in
setup, but test_help fails because the actual output now includes
a description of the new list option. Let’s adjust for that.

We could write a complicated regex to match the entire help output
string, but instead let’s just test parts of the output. We can use
Minitest’s capture_io and assert_match features for that.

A Short Flag Option

We want to be able to use the shorter -l option for listing
currencies, so let’s test that as well.

Test for Short List Flag Option

deftest_short_list_flag

parser.parse!(%w[-l])

assert(options.show_list)

end

Surprisingly this test passes right away, even though we haven’t
specified a short option for --list. OptionParser must be
helping again. I’d feel better if we were explicit about the short
option, so let’s add it.

Specifying the Short List Option

defsetup

super

@options=OpenStruct.new

@parser=OptionParser.newdo|parser|

parser.program_name="PROGRAM"

parser.version="VERSION"

parser.on("-l","--list","Show currency list")do|flag|

options.show_list=flag

end

end

end

The new test is still passing, but our help test fails again. We need
to tweak it a bit for the new help output. The help test seems pretty
brittle; we’ll have to think about that when we’re done learning about
OptionParser.

Other Options

In order to convert currencies with our command-line application, we
might just pass in the amount and source and target currencies without
any option flags. Something like currencyfx 100 USD EUR. Let’s see
how OptionParser handles that.

We’ve been using the parse! method all along, since that’s what the
examples use. The documentation says that parse! is the “[s]ame as
parse, but removes switches destructively. Non-option arguments remain
in argv.”

Test for Extra Options

deftest_non_option_args_left_behind

args=%w[-l FOO BAR BAZ]

parser.parse!(args)

assert_equal(%w[FOO BAR BAZ],args)

end

That test passes right away, so it looks like we understood the
documentation correctly.

Conclusion

We’ve now explored all of the features of OptionParser that we think
we’ll need for the stories we’re working on.

We learned quite a bit about how OptionParser works along the way,
including some surprising built-in behavior that isn’t documented.
This would be an excellent time to file a documentation improvement
PR. I’ll leave that as an exercise for the reader for now.

We can keep these tests as a record of our learning. Or we can use
them as a reference for our actual implementation, then delete them
once we’re testing our real code. We’ve even got a basic
implementation of the OptionParser we’ll need in our real code.

The next time you need to work with an unfamiliar gem or library, try
writing learning tests instead of just poking around in irb.