WTF?

At the first look, specially because of the use of reduce I thought it was a complicated issue to solve.

2 seconds after I realized.

Why using reduce for sum when Python already has the built in sum function?

result = sum([i.thing.price for i in ModelObject.objects.filter(created_at__gte=date)])

Well Python gives us powerful builtins so just use this!

The other problem here is the memory usage of the above solution, it will first get the objects list from filter() and after that it will iterate one by one, doing a field lookup to take the price and return a new list with values to sum.

It can kill your server!

On this case things can be done in a better way! we are talking about Django! and even I am being a Django ORM hater I know that it has some cool things like this one:

Django ORM aggregations

On the above code, the aggregation Sum will translate in to a SQL command and the sum will be performed on the database side! much better

I really do not like the Django ORM syntax, also I hate the way I bind the objects, maybe because I am used to use the wonderful DAL
I prefer to refer to data as data, I mean, data as Rows not data as objects.
But in cases I am working with Django, I think the best is to use its powerful tools!

Revisiting this with a ListField type you can use. But it makes a few of assumptions, such as the fact that you're not storing complex types in your list. For this reason I used ast.literal_eval() to enforce that only simple, built-in types can be stored as members in a ListField:

Recently I tried many ways to add a simple record counter on Django admin home page, I needed it to look like this:

I have tried django admin tools, overwriting the _meta on admin.py but the probleam with admin tools is that it installed a lot of aditional stuff I did not like to use, and
the problem with other approaches was because I needed it to be dynamic. Overwriting the __meta seemed to be the right way but is binded only one time, and no updates done until the app restarts.

When Python runs a script and an uncatched exception is raised, a traceback is printed and the script is terminated. Python2.1 has introduced sys.excepthook, which can be used to override the handling of uncaught exceptions. This allows to automatically start the debugger on an unexpected exception, even if python is not running in interactive mode.

# code snippet, to be included in 'sitecustomize.py'
import sys
def info(type, value, tb):
if hasattr(sys, 'ps1') or not sys.stderr.isatty():
# we are in interactive mode or we don't have a tty-like
# device, so we call the default hook
sys.__excepthook__(type, value, tb)
else:
import traceback, pdb
# we are NOT in interactive mode, print the exception...
traceback.print_exception(type, value, tb)
print
# ...then start the debugger in post-mortem mode.
pdb.pm()
sys.excepthook = info

The above snipper can be easily included in your editor snippets and you can set it on top of your files, or even better you can include on your sitecustomize.py

NOTE:
Use: import pywin.debugger and pywin.debugger.pm()
if you want a gui

TODO

RQ (Redis Queue)

RQ (Redis Queue) is a simple Python library for queueing jobs and processing them in the background with workers. It is backed by Redis and it is designed to have a low barrier to entry. It should be integrated in your web(2py) stack easily.

web2py

Queueing jobs with RQ and web2py

web2py as many other web frameworks works in a request -> response environment, which means that there is a lifetime for things to be done. This lifetime we call "request time", it is the time between the client requests a resource (i.e hits an url of our app or post a form) and the time that the server gives the response back to the client (i.e: The server sends html, json or any other kind of response).

The problem with this is the fact that we have a time-out and the user does not want to wait for tasks to be done, I mean in example for creating image thumbnails, users have to upload a picture and then wait for the thumbnail to be created to have a response from server. Or in the case of sending an email, user fill a contact form and have to wait for the message to be sent. It can take a long time and sometimes it will fail.

The solution is to enqueue that jobs on background and then watch its results to give a response to the user, this response can be given through a websocket or ajax long pooling. (I will not cover this here)

Setting up

install Redis

In debian based linuxes you can do: sudo apt-get install redis-server

Install RQ (redis queue)

sudo pip install rq

case 1 : Sending email in background

User will fill our contact form and then click in submit, instead of sending the email we are going to enqueue the email to be sent by the redis queue.

1. In your models create your Queue object (also you need to have the mail settings)

Start the above worker under web2py environment

With the above worker running the enqueued tasks will be executed and then worker will keep listening for new tasks.

You can also put the worker to run in backgroungm for this you shoud use nohup python web2py.py -S yourappname -M -R /some/path/web2py-rq.py & or even better you can put this to run under the supervidord