README.md

Py2crazy

Py2crazy is a modified version of the official Python interpreter
(CPython 2.7.5) that supports finer-grained tracing and debugging.

It implements the following four features:

Each Python bytecode instruction maps to a range
of line and column numbers in the portion of the source code
that compiled into that instruction.

Debugger applications (such as those built upon bdb)
call the trace function at each executed bytecode rather than
at each executed source line.

Peephole optimizations and opcode prediction macros
are disabled so that source code matches more closely with bytecode.
Doing so makes stepping through executed bytecodes appear
more intuitive, since steps aren't "magically" skipped.

The frame object exposes a new f_valuestack field, which is
a list containing the current values on the expression stack used
by the Python bytecode VM. This field allows debugging and tracing tools
to inspect intermediate results of expression evaluation.

Why would anyone do this?

I created Py2crazy to support finer-grained expression-level tracing
in Online Python Tutor. This wiki
page
discusses some of the design rationale. To illustrate,

Precise line and column info in bytecodes

Here's an illustration of the first (and most significant) feature. If you compile this code

x = 5
y = 13
if (x + 5 > 7) and (y - 3 == 10):
print 'You win'

with regular Python 2.7.5 and disassemble it (python -m dis), you get roughly the following bytecode:

Note that each bytecode instruction maps to one line of source code.

In contrast, if you compile this code with Py2crazy and disassemble, you can see that each bytecode
maps not only to a line, but also to a precise range of columns within that line (highlighted in yellow):

This level of detail makes it possible to create much more fine-grained tracing and debugging tools,
such as expression-level stepping for Online Python Tutor.

Most terminals support colors, so you should be able to see the yellow highlights.

To programmatically access this data, import super_dis and hook into the proper functions from your code.

(Note that super_dis.py works only with Py2crazy, not with regular Python.)

How does Py2crazy debugger stepping differ from regular Python stepping?

Normally, the debugger interface (bdb) registers a tracing function into the
regular CPython interpreter and steps through the target program roughly
one line at a time.

However, when run in Py2crazy, bdb steps one bytecode
instruction at a time, which provides much finer-grained tracing.

To see the difference, run pdb (the standard Python debugger built upon bdb)
on a test file in both regular Python

python -m pdb test.py

and Py2crazy:

Py2crazy/Python-2.7.5/python -m pdb test.py

What did you change in CPython 2.7.5?

Check out the Git repo and run

git diff d36dfc8ffaf5337adb96bd582e0733fe2ffe3f02

to see diffs against a fresh Python 2.7.5 source distribution.

Caveat: Although you might find some ideas in Py2crazy to be useful, its design
is ultimately driven by pedagogical goals, not by industrial-strength
debugging goals. For instance, run-time efficiency wasn't a concern.