Debugging A Hung (Spinning) Python Process

From:
andrew cooke <andrew@...>

Date:
Mon, 31 May 2010 09:46:01 -0400

I left a Python script running at the client's, overnight, expecting
everything to be ready in the morning. But when I checked next day the CPU
was pegged at 100% and the log indicated no progress since 1am. Clearly the
Python code had a bug and was "spinning" in a tight loop.
The problem, then, was to find out what code was being executed. Luckily
there's a very neat solution, that I've used before, that does exactly this.
It's described in an answer by "spiv" at
http://stackoverflow.com/questions/132058/getting-stack-trace-from-a-running-python-application
That describes how to connect gdb to the process ("gdb -p PID") and dump a
Python stack trace ("pystack"). All that you need to do is create a .gdbinit
file with the contents given here -
http://svn.python.org/projects/python/trunk/Misc/gdbinit
With a stack trace it was easy to see the error in my code - and it was a
tight loop, and it depended explicitly on the time of day, which is why I had
not seen it before. :o)
Andrew

Update

From:
andrew cooke <andrew@...>

Date:
Thu, 31 May 2018 21:16:09 -0400

This is now simpler.
You need to follow the instructions at
https://devguide.python.org/gdb/ - basically put
add-auto-load-safe-path /path/to/checkout
in ~/.gdbinit
Then connect with gdb -p PID and use the commands described in the
link.
There is one extra step for VMs. You need to link the python-dgb.py
in the checkout directory to the bin directory for the virtualenv, and
the name MUST MATCH the python in use (so python-3.5, if used, not
simply python)
For example, ths worked for me:
ln -s /home/andrew/pkg/Python-3.5.1/python-gdb.py env/bin/python3.5-gdb.py
Andrew