This PEP proposes a backwards compatible mechanism that allows __enter__()
methods to skip the body of the associated with statement. The lack of
this ability currently means the contextlib.contextmanager decorator
is unable to fulfil its specification of being able to turn arbitrary
code into a context manager by moving it into a generator function
with a yield in the appropriate location. One symptom of this is that
contextlib.nested will currently raise RuntimeError in
situations where writing out the corresponding nested with
statements would not [1].

The proposed change is to introduce a new flow control exception
SkipStatement, and skip the execution of the with
statement body if __enter__() raises this exception.

This PEP was rejected by Guido [4] as it imposes too great an increase
in complexity without a proportional increase in expressiveness and
correctness. In the absence of compelling use cases that need the more
complex semantics proposed by this PEP the existing behaviour is
considered acceptable.

The semantics of the with statement will be changed to include a
new try/except/else block around the call to __enter__().
If SkipStatement is raised by the __enter__() method, then
the main section of the with statement (now located in the else
clause) will not be executed. To avoid leaving the names in any as
clause unbound in this case, a new StatementSkipped singleton
(similar to the existing NotImplemented singleton) will be
assigned to all names that appear in the as clause.

The components of the with statement remain as described in PEP 343[2]:

with EXPR as VAR:
BLOCK

After the modification, the with statement semantics would
be as follows:

mgr = (EXPR)
exit = mgr.__exit__ # Not calling it yet
try:
value = mgr.__enter__()
except SkipStatement:
VAR = StatementSkipped
# Only if "as VAR" is present and
# VAR is a single name
# If VAR is a tuple of names, then StatementSkipped
# will be assigned to each name in the tuple
else:
exc = True
try:
try:
VAR = value # Only if "as VAR" is present
BLOCK
except:
# The exceptional case is handled here
exc = False
if not exit(*sys.exc_info()):
raise
# The exception is swallowed if exit() returns true
finally:
# The normal and non-local-goto cases are handled here
if exc:
exit(None, None, None)

With the above change in place for the with statement semantics,
contextlib.contextmanager() will then be modified to raise
SkipStatement instead of RuntimeError when the underlying
generator doesn't yield.

Currently, some apparently innocuous context managers may raise
RuntimeError when executed. This occurs when the context
manager's __enter__() method encounters a situation where
the written out version of the code corresponding to the
context manager would skip the code that is now the body
of the with statement. Since the __enter__() method
has no mechanism available to signal this to the interpreter,
it is instead forced to raise an exception that not only
skips the body of the with statement, but also jumps over
all code until the nearest exception handler. This goes against
one of the design goals of the with statement, which was to
be able to factor out arbitrary common exception handling code
into a single context manager by putting into a generator
function and replacing the variant part of the code with a
yield statement.

Specifically, the following examples behave differently if
cmB().__enter__() raises an exception which cmA().__exit__()
then handles and suppresses:

With the proposed semantic change in place, the contextlib based examples
above would then "just work", but the class based version would need
a small adjustment to take advantage of the new semantics:

There is currently a tentative suggestion [3] to add import-style syntax to
the with statement to allow multiple context managers to be included in
a single with statement without needing to use contextlib.nested. In
that case the compiler has the option of simply emitting multiple with
statements at the AST level, thus allowing the semantics of actual nested
with statements to be reproduced accurately. However, such a change
would highlight rather than alleviate the problem the current PEP aims to
address: it would not be possible to use contextlib.contextmanager to
reliably factor out such with statements, as they would exhibit exactly
the same semantic differences as are seen with the combined() context
manager in the above example.

Implementing the new semantics makes it necessary to store the references
to the __enter__ and __exit__ methods in temporary variables instead
of on the stack. This results in a slight regression in with statement
speed relative to Python 2.6/3.1. However, implementing a custom
SETUP_WITH opcode would negate any differences between the two
approaches (as well as dramatically improving speed by eliminating more
than a dozen unnecessary trips around the eval loop).