Programming | Python | ML

Primary Menu

We all know how amazingly exhaustive Django debugging can be, when you enable the Debug = True option in your settings file. It gives you a full traceback, complete with the str-ed versions of local variables. However, this post deals with the production-time scenario, when you might encounter an un-anticipated error in your code. Such errors can arise out of various reasons, such as-

1. A kind of input you did not anticipate.

2. A logical error of some kind, usually occuring out of boundary cases (like a sqrt function coming across a negative value)

3. (Worst kind) Some type of low-probability scenario you forgot to test.

In such cases, Django serves the 500 status code along with a “Server Error” message. All this is good, and if your application doesnt have any state variables that may get screwed, or low-probability errors aren’t that important to you, then you can simply ignore the rest of this post. However, if you would want atleast a minimalistic note being made on the server when something goes wrong, then you can set up a simple debug-logging system as a part of your Django project.

Heres how you go about it-

Step 1. Create a folder called debug in your Django project’s main directory (the one that contains the manage.py file).

Step 2. Make an empty __init__.py file inside the new folder, which will tell Python thats it contains relevant code.

Step 3. Make a models.py file in the folder, and add the following lines to it:

The attributes are again to your taste – If you know the basics behind configuring the Django admin page, you already know what you want and how to get there. If you don’t, reading up on the Django tutorial will help.

Step 5. This is where we add the code that makes things work. Add a file called decorators.py in the directory, and add the given code to it:

from debug.models import ExceptionLog
from django.utils import timezone
from django.http import HttpResponse
def log_exceptions(view_name):
"""
Logs all the exceptions occuring in a Django view, to the
ExceptionLog model.
'view_name' denotes an identifier for the view that is
being debug-logged.
"""
def real_decorator(actual_view):
"""
This is the actual decorator.
"""
def wrapped_view(request):
"""
This is the version of the view that will monitor
itself for any un-expected Exception, and maintain basic
logs of the same.
"""
try:
#Run the view code
response = actual_view(request)
#If a response is generated without any Exception
#coming up, return it
return response
except Exception as e:
#If an unexpected Exception occurs, make a debug entry
#and save it
debug_entry = ExceptionLog(
timestamp=timezone.now(),
view=view_name,
exceptionclass=str(e.__class__),
message=str(e))
debug_entry.save()
#Return the Server Error(500) status code
return HttpResponse(status=500)
return wrapped_view
return real_decorator

This code uses the Python ‘magic’ called decorators with arguments. Frankly, I had never implemented Python decorators (used yes, during SymPy work), let alone with arguments, before this. But trust me- unless you have a very special case, this is the way to go for what we want to achieve here.

The decorator basically takes in as argument the name of the view, so that its known during logging. It encapsulates the working of the actual Django view thats being decorated in a try...except block, and logs the time-stamp, view name and the most important Exception details, in the event that one is raised.

Step 6. Add 'debug' to the 'INSTALLED_APPS' tuple in your settings.py Django file. This officially recognizes your new debug-logging app as a part of your complete server code.

Then, before re-starting the server, dont forget to do syncdb or migrate (depending on which Django version you use) on your project. This will essentially register the new models on the server, and make the details appear on your Django admin page.
Step 7. To set up any view in your project code for the debug-logging, just modify it as follows-

If you also use the @csrf_exempt decorator on your view(s), make sure the log_exceptions decorator lies below it.

Voila! You are done! Now, whenever your decorated views raise an Exception, the relevant information will be logged on your Django admin page.

Ofcourse, this does not replace what real debugging does, nor is it as exhaustive (no traceback provided, for starters). It just stores the very basic info about what went wrong, when and where(in terms of which view). However, it will not give you complete details like file name, line no. etc. In a production environment, debugging to that extent is stupid and (should be) unnecessary. That brings me to the second important point-

Whenever an Exception is raised in your Python code, there are two things you can do about it-

1. ‘Duck it’ – Not handle it, letting it propagate down the function stack, OR

2. ‘Handle it’ – Using something like a try...except block, either repair the damage caused by the error (say returning a default value), or raise another Exception that better describes what went wrong (most often, in ‘high level’ terms).

I implore you to do the latter in your code, atleast as much as possible. Why? It just makes your life way easier. In simple programming terms, the lesser the depth of your function stack when an Exception is raised, the easier it is for you to understand what went wrong. But ofcourse, if you already knew things were going to go wrong, you would not need this debugging tool :-P.

But even then, try not to let Exceptions come out from places like the NumPy core, where it becomes frustratingly difficult to know how things screwed up. It is always a good practice to raise good-quality Exceptions yourself, with custom error-messages that say what went wrong. As long as you can understand the problem from looking at the Exception class and message, you are good to go!

EDIT 1: Ofcourse you could use better frameworks meant for this like Sentry and Opbeat. I wrote this post just to show a way to do it quick-and-dirty, in your code itself. If needed, not re-inventing the wheel is ofcourse always better :-).

Ofcourse! That would be one way to go, I wanted a way to have stuff appear on the DJango admin in a seam-less manner. About sending mails everytime theres an error, that might not be the best of doing things in every scenario, for eg sensor networks where your inbox may get flooded with emails if the the same error repeats.

I applaud you for taking this initiative – an important issue in this day and age! It is so important to open the lines of communication early. Once they get to be teenagers, it is much more difficult to get them to open up and share, or even listen. A healthy, open dialog early in life allows them to trust you when they have questions, and once they do get older, they will come to you for advice. I kept the lines of communication open with my 4 and it made all the difference. xx

Sorry, didn’t detailed my answer enough, If you haven’t used sentry, you must take a look. There is pretty cool dashboard with error analysis out of the box (like frequency, counts etc). We also send a mail to our error email. You can always configure it pretty nicely to not spam your email. Also, default django admin is not pretty good at showing stuff. Filtering and searching for stuff would easily get out of hand if you have pretty big database. Having used sentry for almost 8 months on production now, I must say it is pretty nice tool.