Python 2.1 and Nested Scopes

04/19/2001

PythonLabs released Python
2.1 Tuesday. It introduces new magic methods for rich
comparisons, minor language tweaks, and new modules for the standard
library. The most important change, though, is nested scopes.
Because this change is likely to break some code, 2.1 doesn't give you
nested scopes by default. The biggest change in 2.1 isn't really in
2.1. It will be in 2.2. To avoid breaking everyone's code, the
Python developers introduced a __future__ module. If you
want nested scopes, you have to import them from the future:

from __future__ import nested_scopes

Nested scopes confused me at first. Maybe some new programmers can
learn from my mistake, and seasoned programmers can laugh at it. So
for your edification or amusement, let's take a closer look at nested
scopes.

Scope determines which namespace is used to look up a given name.
A namespace is a dictionary of variable, function, and class names.
Any name you might define or assign to is in some namespace. In
Python 2.0, there are, at most, three namespaces visible at any given
time: a local namespace (names defined in the current function), a
global or module namespace (names defined at the top level), and a
built-in namespace (names predefined by Python). Python gives each
function its own local name space. When Python
2.0 tries to find a name, it looks first in the local namespace,
then the global, and finally in the built-in namespace. Some Python
people call this the LGB rule.

The LGB rule works great most of the time. When defining a
function inside a function, however, it can bite you. Say you define
function B inside of function A, and you assigned a few variables in A
that you want to use in function B. Well, in 2.0, B doesn't see
variables assigned in A. Say you assigned a variable named spam in A,
and you want to use it in B. When you try, you get a NameError:
global name 'spam' is not defined. Python just skips right
over A's namespace. All you have is B's local namespace, the global
namespace, and the built-in namespace. There are ways around this. To
get B to see spam in Python 2.0, you might pass it as a default to
your function in your function definition, for example: def
B(spam=spam):. It's ugly, especially when you have several
variables you want B to use, but it works.

Python 2.1 introduces nested scopes. If our hypothetical function
B is defined in A, and uses some name which has not been assigned in
B, Python will now look for that name in the namespace of the
enclosing scope in which it was defined or, in other words, A's
namespace. Note the word "defined". This is where I got confused. To
test out these new rules, I tried something like

You might think with nested scopes you would get both eggs and
bacon with your spam; but you don't, you only get eggs. That's
because where a function is defined determines scope, not
where it is invoked. When I asked about this on
comp.lang.python, Bjorn Pettersen helpfully informed me that
this is the difference between lexical (or static) and dynamic
scoping. Once I had the right names for the concepts, I found the
differences explained in many places on the net. I also finally
understood the difference between my() and local() in Perl, a language
that uses both dynamic and lexical scoping. But I digress. The
following example gets you eggs and bacon with your spam:

Under 2.0, following the LGB rule, you still get eggs. Under 2.1 without the import statement you get eggs and warnings about overwriting global
variables. With nested scopes
you finally get some bacon. Function C gets access
to names in B's namespace.

You will find fancier nested scope tricks in Andrew M. Kuchling's
What's New in Python 2.1,
and even more examples and deeper details in the original Python
Enhancement Proposal (PEP) for nested scopes. The nested scope
change implies some changes in the use of from module import
*, and it may cause havoc in some other cases. Details are in
the PEP. You should start working with nested scopes now and take
advantage of the grace period to fix up your code. In 2.2, nested
scopes will be turned on by default. There will be no from
__past__ import unnested_scopes