Fredrik Håård's Blaag

I'm a programmer, consultant, developer, occasional teacher and
speaker. Among my least disliked programming languages are Python, and
a majority of these posts are related to Python in one way or another.

Python Closures and Decorators (Pt. 2)

In part 1, we looked at sending functions as arguments to other
functions, at nesting functinons, and finally we wrapped a function in
another function. We'll begin this part by giving an example
implementation on the exercise I gave in part 1:

So, this is at least mildly useful, but it'll get better! You may or
may not have heard of closures, and you may have heard any of a
large number of defenitions of what a closure is - I won't go into
nitpicking, but just say that a closure is a block of code (for
example a function) that captures (or closes over) non-local (free)
variables. If this is all gibberish to you, you're probably in need of
a CS refresher, but fear not - I'll show by example, and the concept
is easy enough to understand: a function can reference variables that
are defined in the function's enclosing scope.

For example, take a look at this code:

>>>a=0>>>defget_a():...returna...>>>get_a()0>>>a=3>>>get_a()3

As you can see, the function get_a can get the value of a, and
will be able to read the updated value. However, there is a
limitation - a captured variable cannot be written to:

>>>defset_a(val):...a=val...>>>set_a(4)>>>a3

What happened here? Since a closure cannot write to any captured
variables, a = val actually writes to a local variable a that
shadows the module-level a that we wanted to write to. To get
around this limitation (which may or may not be a good idea), we can
use a container type:

So, with the knowledge that a function captures variables from it's
enclosing scope, we're finally approaching something interesting, and
we'll start by implementing a partial. A partial is an instance of
a function where you have already filled in some or all of the
arguments; let's say, for example that you have a session with
username and password stored, and a function that queries some
backend layer which takes different arguments but always require
credentials. Instead of passing the credentials manually every time,
we can use a partial to pre-fill those values:

>>>#Our 'backend' function...defget_stuff(user,pw,stuff_id):..."""Here we would presumably fetch data using the supplied
... credentials and id"""...print("get_stuff called with user: %s, pw: %s, stuff_id: %s"%(...user,pw,stuff_id))>>>defpartial(fn,*args,**kwargs):...deffn_part(*fn_args,**fn_kwargs):...kwargs.update(fn_kwargs)...returnfn(*args+fn_args,**kwargs)...returnfn_part...>>>my_stuff=partial(get_stuff,'myuser','mypwd')>>>my_stuff(3)get_stuffcalledwithuser:myuser,pw:mypwd,stuff_id:3>>>my_stuff(67)get_stuffcalledwithuser:myuser,pw:mypwd,stuff_id:67

Partials can be used in numerous places to remove code duplication
where a function is called in different places with the same, or
almost the same, arguments. Of course, you don't have to implement it
yourself; just do from functools import partial.

Finally, we'll take a look at function decorators (there may be a post
on class decorators in the future). A function decorator is (can be
implemented as) a function that takes a function as parameter and
returns a new function. Sounds familiar? It should, because we've
already implemented a working decorator: our print_call function
is ready to be used as-is:

But what if we want to be able to parameterize the decorator? In this
case, the function used as a decorator will received the arguments,
and will be expected to return a function that wraps
the decorated function:

>>>defrequire(role):...defwrapper(fn):...defnew_fn(*args,**kwargs):...ifnotroleinkwargs.get('roles',[]):...print("%s not in %s"%(role,kwargs.get('roles',[])))...raiseException("Unauthorized")...returnfn(*args,**kwargs)...returnnew_fn...returnwrapper...>>>@require('admin')...defget_users(**kwargs):...return('Alice','Bob')...>>>get_users()adminnotin[]Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>File"<stdin>",line7,innew_fnException:Unauthorized>>>get_users(roles=['user','editor'])adminnotin['user','editor']Traceback(mostrecentcalllast):File"<stdin>",line1,in<module>File"<stdin>",line7,innew_fnException:Unauthorized>>>get_users(roles=['user','admin'])('Alice','Bob')

...and there you have it. You are now ready to write decorators, and
perhaps use them to write aspect-oriented Python; adding @cache,
@trace, @throttle are all trivial (and before you add @cache, do check
functools once more if you're using Python 3!).