Fixtures

This post will cover the basics of pytest fixtures and how we use them in h.

A “test fixture” is some code that needs to be run in order to set things up
for a test. When we created the objects that we needed to pass into the
method we were testing in the arrange part of the
arrange, act, assert recipe, those were fixtures.
Test factories are helpers for easily creating fixtures.
Fixtures don’t have to be objects that we pass into the method we’re testing
as arguments, they can be anything that needs to be set up for a test.
For example if we want to test what a method does when there are three
documents in the database, setting up the db with those three documents is a
fixture.

Setup and teardown methods

Most unit testing frameworks, for example
the standard unittest library that ships with Python,
use the familiar setup() and teardown() methods to set up all the necessary
fixtures before running a test method and, if necessary, tear things down after
running the test (before running the next test). The test code might look
something like this:

The test framework runs the setup() method once before it runs each test
method (and the teardown() method, if there is one, gets run once after each
test).

The drawbacks of setup() methods begin to show when a test class contains
different test methods that use different fixture objects or setup:

classMyTestClass(object):defsetup(self):self.first_fixture_object=SomeClass()self.second_fixture_object=SomeOtherClass()deftest_something(self):# Some test code that uses self.first_fixture_objectdeftest_something_else(self):# Some test code that uses self.second_fixture_object...

Some of the test methods only need first_fixture_object, and some
only need second_fixture_object, but both fixture
objects get created for each test. In a small example this isn’t important but
over a large test base unnecessarily running a lot of unused fixtures for a lot
of tests can start to add significantly to how long it takes to run the tests
every time any developer needs to do so.

As they get bigger and more complicated tests like this can also become hard
to read. A whole lot of setup is done in the setup() method, a lot of which
may be depended upon by the test method that you’re currently trying to
understand, and a lot of it not. You have to scan the whole test method body
looking for all the self.somethings to see what fixtures this test depends
on, and then look in setup() to see what each of these fixtures is.

Worse, if a fixture sets up something in the environment, for example adding
some objects to the test database, then a test method may be depending on that
fixture (assuming that certain objects exist in the db) without explicitly
referencing that fixture in the test body at all.

It can quickly get to the point where setup() code can be difficult to change
without potentially breaking multiple test methods for unclear reasons, and
where it can become hard to tell exactly what a given test method is really
testing.

The problem gets worse when different test fixtures conflict with each other.
This can happen when there’s a singleton resource such as a database.
If one test needs to test what happens when there are no users in the database,
and another needs to test what happens when there are three users in the
database, well a setup() method can’t setup the database with zero users in
it and also setup the same database with three users in it as the same time.
Hacking around this kind of issue can lead to hard to read code and some nasty
coupling between test methods.

One solution would be to break the methods up into multiple test classes.
You think of each test class as its own fixture. If some tests need no users
and one group in the db then that’s one fixture and one test class, if some
other tests need to test what happens when there’s three users and no groups in
the db then that’s another class:

classTestWithNoUsersAndOneGroup(object):defsetup(self):# Add one group to the test db....deftest_something(self):...deftest_something_else(self):...classTestWithThreeUsersAndNoGroup(object):defsetup(self):# Add three users to the test db....deftest_another_thing(self):...

You may end up with some code duplication between the different setup()
methods, but that can always be moved into helper methods that each of the
setup() methods calls.

Clearly coming up with good names for the test classes can get pretty awkward
if the different fixtures that they represent become complex and varied.

In practice, test classes hardly ever get written like this.
Most commonly you see test classes named things like TestFoo containing all of
the tests for the Foo class (or perhaps the foo() method),
and this is how test classes are mostly used in h as well - to group together
the tests for a given unit of code that’s being tested.

Another alternative is to use test helper methods…

Test helper methods

The problems with setup() methods can be worked around by simply not usingsetup() methods or self, and instead using simple helper methods -
non-test methods in the test classes or modules that are called by each test
method as needed:

classMyTestClass(object):defadd_one_group_to_the_db(self):# Code that adds one group to the db and returns the group.defadd_three_users_to_the_db(self):# Code that adds three users to the db and returns the users/deftest_something(self):group=add_one_group_to_the_db()...deftest_something_else(self):group=add_one_group_to_the_db()...deftest_another_thing(self):users=add_three_users_to_the_db()...

This works just fine:

Each test calls only the helper methods that test needs.
No helper methods are run unnecessarily for any tests that don’t need them.

It’s easy to look at a test and see exactly what setup is done for that test
and what isn’t, no more implicit dependencies on things that are done or
not done in a setup() method.

The tests are independent from each other, modifying a test or adding
a new test isn’t going to break existing tests. If a new test wants to
change what a helper method does, but doing so would break existing tests,
you can always just add a new helper method and call that instead.

But simple helper methods like this do have a couple of drawbacks:

Each test needs to call each helper method that it needs, which clutters up
the test bodies with a lot of repetitive, boilerplate code (this gets worse
when helper methods require arguments to be passed to them).

There isn’t a good way to do any tearing down of something that was set up
by a call to a helper method (you wouldn’t want each test method to have
to call corresponding teardown methods).

Pytest comes with a feature that it calls pytest fixtures that solve this
problem very nicely. Pytest fixtures are like test helper methods on
steroids…

Pyramid is the web framework that h uses to receive
HTTP requests from users’ web browsers and send back HTTP responses to those
requests. The request object
is the object that Pyramid makes available to our code that represents the HTTP
request that we’re currently responding to. Many methods in h require the
request object as an argument, so tests often need a request object to pass
to the method they’re testing.

The pyramid_request() function above creates a DummyRequest
object, sets various fields on the request that are needed for h, and returns
it.

The @pytest.fixture decorator on the top of the function registers it with
pytest as a fixture function, which means that a test can make use of it by
simply taking it as an argument. For example, let’s look
at a test for the paginate() function:

deftest_current_page_defaults_to_1(pyramid_request):"""If there's no 'page' request param it defaults to 1."""pyramid_request.params={}page=paginate(pyramid_request,600,10)assertpage['cur']==1

paginate()
is a function that takes the request and returns some data, including the
current page, that’s used to render this pagination control at the bottom of
search results pages:

The test needs a Pyramid request argument in order to call the paginator
function, so it adds the pyramid_request fixture to its arguments:

def test_current_page_defaults_to_1(self, pyramid_request):
...

When pytest comes to run this test it’ll see the pyramid_request argument,
it knows that this matches up to an @pytest.fixture function, so it first
calls the pyramid_request() fixture function and then passes
the value that the fixture function returns to the test as the
pyramid_request argument. What pytest does is something like this:

The pyramid_request() fixture returns a Pyramid DummyRequest object all set
up for use in h, and pytest passes that DummyRequest to the test as the
pyramid_request argument. It’s as simple as that - all the test has to do
is have an argument with the same name as the fixture function and it gets the
request object passed right into it.

At its core, that’s all a pytest fixture is - a perfectly normal Python
function that pytest runs before running a test, only if that test has the
fixture as an argument. An @pytest.fixture decorator is needed on the top of
the function to register it with pytest as a fixture.

Fixtures are no different to test helper methods, except the boilerplate of
each test needing to call every helper method that it needs is avoided.
It’s not often needed, but a fixture can also contain teardown code to be
called after any test that used that fixture simply by
using yield instead of return.

Lots of different tests can reuse the same fixture

Just like different tests can call the same helper method, once you’ve
written a fixture once you can use it in as many tests as you want just by
adding it as an argument to each test:

One test can use lots of different fixtures

And of course just like a test can call multiple helper methods, a test can
also use multiple fixtures by just having all of the fixture names as
arguments.

In the factories post we looked at the factories object
which can be used to easily create realistic objects for use in tests.
factories is a fixture
that’s used by any test that wants to use it to create test objects.

pyramid_settings
is another fixture that returns the settings used to configure the h app in the
test environment.

A test method can use all three fixtures, pyramid_request, factories and
pyramid_settings just by taking all three as arguments by name:

Fixtures don’t have to return anything

Sometimes a test just needs a fixture function to be run before the test,
because the test depends on some setup that the fixture function does,
but the test doesn’t actually need to use the value that the fixture function
returns.
A fixture function doesn’t even have to return anything, it might just do some
setup that’s needed by some tests, for example creating some entries in the
database, and return nothing. If a test wants that fixture function to be run
before it then it adds the fixture as an argument, if it doesn’t then it
doesn’t. A good example of this is the routes fixture, which is found in
many places in the h tests.

Many methods in h call methods like route_url() and others that require all
the routes to be setup in order to work. If the routes aren’t configured then
calls to route_url() will crash with some sort of “unknown route” error.
So the tests need some code to setup the routes before the test.
This is what a routes fixture does, for example:

The fixture doesn’t return anything, but it adds the 'activate' route to the
Pyramid config so that any calls to request.route_url("activate", ...) will
now work. A test can use this fixture and then test a function that uses this
route without having it crash:

@pytest.mark.usefixtures

The test above doesn’t actually use the routes argument in the body of the
test, and the value of routes is just None anyway, it’s just there to tell
pytest to call the routes() function before it calls test_something().

@pytest.mark.usefixtures()
is an alternative way for a test to use a fixture when it just needs the fixture
function to be run, but doesn’t actually need the fixture value in the test
body:

Where to find fixtures

When you see test methods that have arguments in h,
unless the test is using parametrize,
those arguments are probably fixtures.
But where do you find the fixture functions for the fixtures that a test uses?
And where should you put any new fixtures that you want to write for your tests?
Fixtures can be defined in a number of places:

Fixtures defined in a test file are only available to the tests in that file.

A fixture in a test file will override any fixture with the same name in a
conftest.py file.

Fixture functions can go in test classes. A fixture defined in a class is
only available to the tests in that class, and overrides any fixture with
the same name defined outside of the class or in a conftest.py file.

We use this a lot in h as well, if a fixture is only needed by the tests in
one test class then we define the fixture as a method of that test class.
It keeps related code together, reducing noise and travel. It also allows
other test classes to have their own fixture with the same name without
conflicting.

Conclusion

So far we’ve seen that fixtures are a more convenient alternative to test
helper methods, with less boilerplate required and with a good way to do
teardown code. You define each fixture in a separate function, but rather than
each test needing to call each helper method that it wants the tests can just
receive the fixture objects as arguments.

Pytest fixtures also have a couple of advanced features that take them well
beyond what simple helper methods can do. We’ll cover some of those in the
next post…