I have a @login_required decorator and a function assert_login_required that checks if a view has been decorated with it. I know the test is quite small but I decided to do some metaprogramming and simplify it even further:

import myapp.views
ensure_login_required(myapp.views.my_view)

Here is the code for ensure_login_required:

import sys
def ensure_login_required(view):
"""
Generate a test function checking that 'view' is using @login_required
"""
def test_func():
assert_login_required(view)
# get global namespace from where this function is being called, if I use
# globals() here I'll actually get the global namespace of this module
namespace = sys._getframe(1).f_globals
namespace['test_%s_requires_login' % view.__name__] = test_func

Yesterday I had this specific scenario where I needed just this: having JSON document with numbers and decode them into my struct with string fields. I found the solution in stackoverflow (of course). encoding/json has a Number type which stores the value as a string and has methods to convert to integer or floats. Here is how I used it:

I still had to change my struct field from string to json.Number, but that’s ok because I can still assign string literals to it. By the way that’s an important distinction. string and json.Number are different types with the same underlying type. From the spec:

Each type T has an underlying type: If T is one of the predeclared boolean, numeric, or string types, or a type literal, the corresponding underlying type is T itself. Otherwise, T’s underlying type is the underlying type of the type to which T refers in its type declaration.

this is the entry point of my library. I have an executable that just parses a directory name from command line and call this function. It’s purpose it to collect all files from a directory, filter based on some rules, and publish the file names in a queue which will be consumed by another script. It runs forever because newly created files are detected and collected too.

Anyway, what it does doesn’t really matter, the problem is: how to test this function since it’s supposed to run forever?

I don’t mind to have just unit tests for this function because I already have more integration-like tests for the classes it uses. The first solution I thought was something like this:

defcollect(directory):sequential=Sequential(directory)live=Live(directory)whileshould_continue():# new function to mock on tests: UGLY!live.process()sequential.process()defshould_continue():returnTrue

this way I could mock should_continue() in my test and make it return False to abort the loop when I want. That works but it’s ugly! I don’t like to add dependency injections only for tests.

I asked on #python on irc and marienz gave a neat idea: raise an exception.
I could mock live.process() and sequential.process() and raise an exception, this way I know they were called as I expected and also it will also abort the loop!

importpytestimportmock# this is the library under testimportcollectors# replace original classed with mock objects@mock.patch('collectors.Sequential')@mock.patch('collectors.Live')deftest_collect_should_loop_forever_processing_both_collectors(collectors_Live,collectors_Sequential):# build mock instances. process() method will raise error# when called for the 2nd time. The code for `ErrorAfter`# is bellowseq=mock.Mock(['process'])seq.process.side_effect=ErrorAfter(2)live=mock.Mock(['process'])live.process.side_effect=ErrorAfter(2)# ensure mocked classes builds my mocked instancescollectors_Sequential.return_value=seqcollectors_Live.return_value=live# `ErrorAfter` will raise `CallableExhausted`withpytest.raises(CallableExhausted):collectors.collect('/tmp/files')# make sure our classed are instantiated with directorycollectors_Sequential.assert_called_once_with('/tmp/files')collectors_Live.assert_called_once_with('/tmp/files')

This test uses py.test and mock. I hope the comments explains enough. The idea is simple: make process() raise an Exception to abort the loop.
The ErrorAfter class is a small helper, it builds a callable object that will raise a specific exception after n calls. I created a custom exception here to make sure my test fails if any other exception is raised. See the code bellow.

Conclusion

Try to avoid as much as possible to create dependency injections specifically for your tests. In dynamic languages like Python it’s very easy to replace a specific component with a mock object without adding extra complexity to your code just to allow unit testing.

This was the first time I had to test a infinite loop, it’s possible and easy!

Last week I wrote a post about unnecessary classes, and that kept me thinking about how common people write classes or more complex designs upfront just because it would be easier to evolve and extend later.

One suggestion I gave was to use a dictionary instead of a class which had no methods but the constructor. This class was just holding state, data, so a dictionary is enough. Let’s start simple…

I’ll stick with this example because it’s real code, even if it’s purpose is not 100% clear:

the problem now is that button_links are being used as dictionaries all over the place, like button_link['size']. And even worse, this could be a public API and you may not have access to the clients using it. In this case we could simulate the dict api implementing __getitem__:

importwarningsclassButtonLink(object):# ... same as abovedef__getitem__(self,item):warnings.warn("Dict-like access is deprecated, please use `.{0}`".format(item),DeprecationWarning)returngetattr(self,item)

Now you can use button_link.size and button_link['size']. And I also added a deprecation warning to notify the users that they should use the object API from now on.
And you should also modify the functions above to create and manipulate the object instead of the dictionary, also raising warnings if you want.

You may want to implement other dictionary methods, like keys, items, has_key, __contains__, etc.

Why not make a dict subclass?

The main reason I wouldn’t subclass dict in this case is philosophical: ButtonLink should behave like a dict (temporarily), but it is not a dictionary. A good example of a dict subclass is OrderedDict, it is a dictionary with customized behavior.

Another aspect to keep in mind is when we write a subclass is that we inherit all the methods from the superclass, all the API, and it’s now part of my class API. In my case I don’t want to implement the __setitem__ method, because the clients should not be using it, all the time you need to modify the button_link dict there is a specialized function to do so, and these functions I can easily rewrite to use the object API.

“What about isinstance()?!”. Yeah, you’re going to loose it, since button_link is not a dict anymore isinstance(button_link, dict) is False. But I believe you should not be using it, type checking with isinstance() and even worse, type(), is not very common on languages like python. “If it walks like a duck and quacks like a duck, then it is a duck”! I know that sometimes isinstance() is handy, but I don’t think it’s appropriate here because of the philosophical argument I gave above.

Conclusion

Start simple, it’s always easier to evolve to a more complex design than the other way around.

Using Python magic methods your objects can easily look like a native data structure, so start using simple data structures and write your own if you need later. Another language feature that helps this incremental design evolution is properties, a simple attribute can evolve to a complex getter/setter transparently.

Today I opened a python module that made me feel sad. It has two classes: one of them has 4 methods, all of them static, and no attributes; the other has only one method: __init__, and 5 attributes.

Why people still write classes like these?

Use functions

…instead of classes with just static methods. If you have this class just to group the functions in a common namespace then create a new module (.py file), this is how python organize namespaces, and it’s good, use it!

One rule I follow when using dictionaries like the example above is to always modify them with specialized functions. You’ll end up with well defined structures and modules that know how to build and manage these structures inside your application.

What I’m suggesting here is actually the opposite of OO, instead of writing a class with state and methods, keep the state on dictionaries (or tuples, lists, sets) and write functions to manipulate the state. I’ve been using this approach much more that classes lately.

References

Stop writing classes is a great talk by Jack Diederich at PyCon 2012, showing examples where classes are overused.

Ned Batchelder committed a new feature to coverage.py 2 days ago, it’s a way to verify coverage threshold. Basically a way to fail your tests if the coverage is not enough.For now you need to install from the repository to get the new feature

If your coverage is not 100% your exit status will be 2. This will make your CI fail if your coverage is not enough You may also verify coverage using the python API, in this case you need to verify the return value from report() function, here is an example:

Ned Batchelder committed a new feature to coverage.py 2 days ago, it’s a way to verify coverage threshold. Basically a way to fail your tests if the coverage is not enough.
For now you need to install from the repository to get the new feature

If your coverage is not 100% your exit status will be 2. This will make your CI fail if your coverage is not enough :).
You can also verify coverage using the python API, in this case you need to verify the return value from report() function, here is an example:

I’m working on a small django application these days which offers templates to override django-filebrowser’s templates to remove the Grappelli dependency.

It’s a small app, consisting 99% of templates. There are only 19 tests which take 35 seconds to execute! All of them performs at least one request using django test client, no selenium, no real HTTP requests, and still 35 seconds!

After a run with cProfile I’ve noticed many calls to hash functions, specifically django.utils.crypto.pbkdf2() and it’s helper _fast_hmac. Then I came up with this very complex patch to my settings:

You probably want a safer hash function for your user passwords in production, but there is no problem to use a simpler one for tests. This extensible way to deal with password hashers was introduced in Django 1.4, you can read more about it on the docs.