The standard exception hierarchy is an important part of the Python
language. It has two defining qualities: it is both generic and
selective. Generic in that the same exception type can be raised
- and handled - regardless of the context (for example, whether you are
trying to add something to an integer, to call a string method, or to write
an object on a socket, a TypeError will be raised for bad argument types).
Selective in that it allows the user to easily handle (silence, examine,
process, store or encapsulate...) specific kinds of error conditions
while letting other errors bubble up to higher calling contexts. For
example, you can choose to catch ZeroDivisionErrors without affecting
the default handling of other ArithmeticErrors (such as OverflowErrors).

This PEP proposes changes to a part of the exception hierarchy in
order to better embody the qualities mentioned above: the errors
related to operating system calls (OSError, IOError, mmap.error,
select.error, and all their subclasses).

While some of these distinctions can be explained by implementation
considerations, they are often not very logical at a higher level. The
line separating OSError and IOError, for example, is often blurry. Consider
the following:

The same error condition (a non-existing file) gets cast as two different
exceptions depending on which library function was called. The reason
for this is that the os module exclusively raises OSError (or its
subclass WindowsError) while the io module mostly raises IOError.
However, the user is interested in the nature of the error, not in which
part of the interpreter it comes from (since the latter is obvious from
reading the traceback message or application source code).

In fact, it is hard to think of any situation where OSError should be
caught but not IOError, or the reverse.

A further proof of the ambiguity of this segmentation is that the standard
library itself sometimes has problems deciding. For example, in the
select module, similar failures will raise select.error, OSError
or IOError depending on whether you are using select(), a poll object,
a kqueue object, or an epoll object. This makes user code uselessly
complicated since it has to be prepared to catch various exception types,
depending on which exact implementation of a single primitive it chooses
to use at runtime.

As for WindowsError, it seems to be a pointless distinction. First, it
only exists on Windows systems, which requires tedious compatibility code
in cross-platform applications (such code can be found in Lib/shutil.py).
Second, it inherits from OSError and is raised for similar errors as OSError
is raised for on other systems. Third, the user wanting access to low-level
exception specifics has to examine the errno or winerror attribute
anyway.

Note

Appendix B surveys the use of the various exception types across
the interpreter and the standard library.

The current variety of OS-related exceptions doesn't allow the user to filter
easily for the desired kinds of failures. As an example, consider the task
of deleting a file if it exists. The Look Before You Leap (LBYL) idiom
suffers from an obvious race condition:

if os.path.exists(filename):
os.remove(filename)

If a file named as filename is created by another thread or process
between the calls to os.path.exists and os.remove, it won't be
deleted. This can produce bugs in the application, or even security issues.

Therefore, the solution is to try to remove the file, and ignore the error
if the file doesn't exist (an idiom known as Easier to Ask Forgiveness
than to get Permission, or EAFP). Careful code will read like the following
(which works under both POSIX and Windows systems):

This is a lot more to type, and also forces the user to remember the various
cryptic mnemonics from the errno module. It imposes an additional
cognitive burden and gets tiresome rather quickly. Consequently, many
programmers will instead write the following code, which silences exceptions
too broadly:

try:
os.remove(filename)
except OSError:
pass

os.remove can raise an OSError not only when the file doesn't exist,
but in other possible situations (for example, the filename points to a
directory, or the current process doesn't have permission to remove
the file), which all indicate bugs in the application logic and therefore
shouldn't be silenced. What the programmer would like to write instead is
something such as:

Reworking the exception hierarchy will obviously change the exact semantics
of at least some existing code. While it is not possible to improve on the
current situation without changing exact semantics, it is possible to define
a narrower type of compatibility, which we will call useful compatibility.

For this we first must explain what we will call careful and careless
exception handling. Careless (or "naïve") code is defined as code which
blindly catches any of OSError, IOError, socket.error,
mmap.error, WindowsError, select.error without checking the errno
attribute. This is because such exception types are much too broad to signify
anything. Any of them can be raised for error conditions as diverse as: a
bad file descriptor (which will usually indicate a programming error), an
unconnected socket (ditto), a socket timeout, a file type mismatch, an invalid
argument, a transmission failure, insufficient permissions, a non-existent
directory, a full filesystem, etc.

(moreover, the use of certain of these exceptions is irregular; Appendix B
exposes the case of the select module, which raises different exceptions
depending on the implementation)

Careful code is defined as code which, when catching any of the above
exceptions, examines the errno attribute to determine the actual error
condition and takes action depending on it.

Then we can define useful compatibility as follows:

useful compatibility doesn't make exception catching any narrower, but
it can be broader for careless exception-catching code. Given the following
kind of snippet, all exceptions caught before this PEP will also be
caught after this PEP, but the reverse may be false (because the coalescing
of OSError, IOError and others means the except clause throws
a slightly broader net):

try:
...
os.remove(filename)
...
except OSError:
pass

useful compatibility doesn't alter the behaviour of careful
exception-catching code. Given the following kind of snippet, the same
errors should be silenced or re-raised, regardless of whether this PEP
has been implemented or not:

The rationale for this compromise is that careless code can't really be
helped, but at least code which "works" won't suddenly raise errors and
crash. This is important since such code is likely to be present in
scripts used as cron tasks or automated system administration programs.

Careful code, on the other hand, should not be penalized. Actually, one
purpose of this PEP is to ease writing careful code.

The first step of the resolution is to coalesce existing exception types.
The following changes are proposed:

alias both socket.error and select.error to OSError

alias mmap.error to OSError

alias both WindowsError and VMSError to OSError

alias IOError to OSError

coalesce EnvironmentError into OSError

Each of these changes doesn't preserve exact compatibility, but it does
preserve useful compatibility (see "compatibility" section above).

Each of these changes can be accepted or refused individually, but of course
it is considered that the greatest impact can be achieved if this first step
is accepted in full. In this case, the IO exception sub-hierarchy would
become:

Not only does this first step present the user a simpler landscape as
explained in the rationale section, but it also allows for a better
and more complete resolution of Step 2 (see Prerequisite).

The rationale for keeping OSError as the official name for generic
OS-related exceptions is that it, precisely, is more generic than IOError.
EnvironmentError is more tedious to type and also much lesser-known.

The survey in Appendix B shows that IOError is the dominant
error today in the standard library. As for third-party Python code,
Google Code Search shows IOError being ten times more popular than
EnvironmentError in user code, and three times more popular than OSError
[3]. However, with no intention to deprecate IOError in the middle
term, the lesser popularity of OSError is not a problem.

Since WindowsError is coalesced into OSError, the latter gains a winerror
attribute under Windows. It is set to None under situations where it is not
meaningful, as is already the case with the errno, filename and
strerror attributes (for example when OSError is raised directly by
Python code).

The following paragraphs outline a possible deprecation strategy for
old exception names. However, it has been decided to keep them as aliases
for the time being. This decision could be revised in time for Python 4.0.

Deprecating the old built-in exceptions cannot be done in a straightforward
fashion by intercepting all lookups in the builtins namespace, since these
are performance-critical. We also cannot work at the object level, since
the deprecated names will be aliased to non-deprecated objects.

A solution is to recognize these names at compilation time, and
then emit a separate LOAD_OLD_GLOBAL opcode instead of the regular
LOAD_GLOBAL. This specialized opcode will handle the output of a
DeprecationWarning (or PendingDeprecationWarning, depending on the policy
decided upon) when the name doesn't exist in the globals namespace, but
only in the builtins one. This will be enough to avoid false positives
(for example if someone defines their own OSError in a module), and
false negatives will be rare (for example when someone accesses OSError
through the builtins module rather than directly).

The above approach cannot be used easily, since it would require
special-casing some modules when compiling code objects. However, these
names are by construction much less visible (they don't appear in the
builtins namespace), and lesser-known too, so we might decide to let them
live in their own namespaces.

The second step of the resolution is to extend the hierarchy by defining
subclasses which will be raised, rather than their parent, for specific
errno values. Which errno values is subject to discussion, but a survey
of existing exception matching practices (see Appendix A) helps us
propose a reasonable subset of all values. Trying to map all errno
mnemonics, indeed, seems foolish, pointless, and would pollute the root
namespace.

Furthermore, in a couple of cases, different errno values could raise
the same exception subclass. For example, EAGAIN, EALREADY, EWOULDBLOCK
and EINPROGRESS are all used to signal that an operation on a non-blocking
socket would block (and therefore needs trying again later). They could
therefore all raise an identical subclass and let the user examine the
errno attribute if (s)he so desires (see below "exception
attributes").

Prerequisite, because some errnos can currently be attached to different
exception classes: for example, ENOENT can be attached to both OSError and
IOError, depending on the context. If we don't want to break useful
compatibility, we can't make an except OSError (or IOError) fail to
match an exception where it would succeed today.

Loose, because we could decide for a partial resolution of step 2
if existing exception classes are not coalesced: for example, ENOENT could
raise a hypothetical FileNotFoundError where an IOError was previously
raised, but continue to raise OSError otherwise.

The dependency on step 1 could be totally removed if the new subclasses
used multiple inheritance to match with all of the existing superclasses
(or, at least, OSError and IOError, which are arguable the most prevalent
ones). It would, however, make the hierarchy more complicated and
therefore harder to grasp for the user.

PermissionError: trying to run an operation without the adequate access
rights - for example filesystem permissions (EACCES, EPERM)

BlockingIOError: an operation would block on an object (e.g. socket) set
for non-blocking operation (EAGAIN, EALREADY, EWOULDBLOCK, EINPROGRESS);
this is the existing io.BlockingIOError with an extended role

BrokenPipeError: trying to write on a pipe while the other end has been
closed, or trying to write on a socket which has been shutdown for writing
(EPIPE, ESHUTDOWN)

InterruptedError: a system call was interrupted by an incoming signal
(EINTR)

ConnectionError: a base class for ConnectionAbortedError,
ConnectionRefusedError and ConnectionResetError

The following drawing tries to sum up the proposed additions, along with
the corresponding errno values (where applicable). The root of the
sub-hierarchy (OSError, assuming Step 1 is accepted in full) is not
shown:

Various naming controversies can arise. One of them is whether all
exception class names should end in "Error". In favour is consistency
with the rest of the exception hiearchy, against is concision (especially
with long names such as ConnectionAbortedError).

In order to preserve useful compatibility, these subclasses should still
set adequate values for the various exception attributes defined on the
superclass (for example errno, filename, and optionally
winerror).

Since it is proposed that the subclasses are raised based purely on the
value of errno, little or no changes should be required in extension
modules (either standard or third-party).

The first possibility is to adapt the PyErr_SetFromErrno() family
of functions (PyErr_SetFromWindowsErr() under Windows) to raise the
appropriate OSError subclass. This wouldn't cover, however, Python
code raising OSError directly, using the following idiom (seen in
Lib/tempfile.py):

raise IOError(_errno.EEXIST, "No usable temporary file name found")

A second possibility, suggested by Marc-Andre Lemburg, is to adapt
OSError.__new__ to instantiate the appropriate subclass. This has
the benefit of also covering Python code such as the above.

Making the exception hierarchy finer-grained makes the root (or builtins)
namespace larger. This is to be moderated, however, as:

only a handful of additional classes are proposed;

while standard exception types live in the root namespace, they are
visually distinguished by the fact that they use the CamelCase convention,
while almost all other builtins use lowercase naming (except True, False,
None, Ellipsis and NotImplemented)

An alternative would be to provide a separate module containing the
finer-grained exceptions, but that would defeat the purpose of
encouraging careful code over careless code, since the user would first
have to import the new module instead of using names already accessible.

While this is the first time such as formal proposal is made, the idea
has received informal support in the past [1]; both the introduction
of finer-grained exception classes and the coalescing of OSError and
IOError.

The removal of WindowsError alone has been discussed and rejected
as part of another PEP [2], but there seemed to be a consensus that the
distinction with OSError wasn't meaningful. This supports at least its
aliasing with OSError.

One source of trouble has been with the respective constructors of OSError
and WindowsError, which were incompatible. The way it is solved is by
keeping the OSError signature and adding a fourth optional argument
to allow passing the Windows error code (which is different from the POSIX
errno). The fourth argument is stored as winerror and its POSIX
translation as errno. The PyErr_SetFromWindowsErr* functions have
been adapted to use the right constructor call.

A slight complication is when the PyErr_SetExcFromWindowsErr* functions
are called with OSError rather than WindowsError: the errno
attribute of the exception object would store the Windows error code (such
as 109 for ERROR_BROKEN_PIPE) rather than its POSIX translation (such as 32
for EPIPE), which it does now. For non-socket error codes, this only occurs
in the private _multiprocessing module for which there is no compatibility
concern.

Note

For socket errors, the "POSIX errno" as reflected by the errno module
is numerically equal to the Windows Socket error code
returned by the WSAGetLastError system call:

This PEP ignores EOFError, which signals a truncated input stream in
various protocol and file format implementations (for example GzipFile).
EOFError is not OS- or IO-related, it is a logical error raised at
a higher level.

This PEP also ignores SSLError, which is raised by the ssl module
in order to propagate errors signalled by the OpenSSL library. Ideally,
SSLError would benefit from a similar but separate treatment since it
defines its own constants for error types (ssl.SSL_ERROR_WANT_READ,
etc.). In Python 3.2, SSLError is already replaced with socket.timeout
when it signals a socket timeout (see issue 10272).

Endly, the fate of socket.gaierror and socket.herror is not settled.
While they would deserve less cryptic names, this can be handled separately
from the exception hierarchy reorganization effort.

VMSError is completely unused by the interpreter core and the standard
library. It was added as part of the OpenVMS patches submitted in 2002
by Jean-François Piéronne [4]; the motivation for including VMSError was that
it could be raised by third-party packages.