At the end of this post you will find the source code of a python sandbox that I write for have a general sandbox that work for all my celery tasks.
Anyway this is a general python code and you can use it also without Celery but here I will give you also some example to how to use it in easy and fast way with Celery.

The sandbox is wrote as a Context Manager (under suggestion of asksol 😉 ) with an entry point that will create the folders and an exit point that will clean it.
The class create a folder structure like this:

/sandboxradix/tasks.mytask.dosomething/taskpid/

On the exit point only the taskpid folder will be erased, in this way you can launch several tasks of the same type at the same time
Let’s see how you can use it for you celery task.

In my personal opinion the best way to do it is to create a base class for all your tasks that extend the celery task class and inside it implement some general useful methods like the sandbox behaviour.
Following you can see an example of a general base class that use my sandbox context manager.

As you can read on the comments the __call__ method is called before the task execution.
Here we call the constructor of our sanbox but we don’t initialize it because maybe not all our tasks needs sandbox 😉 .
For the same reason in the method after_return, called form celery at the end of a task whatever is the status (RETRY,FAILURE, ecc.), we will clean the sandbox only if we used it.
The settings module that you see in my general task is the django settings that I use for store conf vars from my task like the PATH_SANDBOX the radix folder for your sandbox.

Now you can write a task by extend the base task that we created and maybe it will look like this

from toforgeTask import toforgeTask
class sampleDownloader(toforgeTask):
def run(self, album_id, **kwargs):
self.logger.info("Download a test file %s " % (album_id))
#: initialize tha sanbox
self.sandbox.__enter__()
#: download a file
self.writeFileFromUrl(self,"http://www.toforge.com/notexistfile.zip",
"myfile")
#: do some stuff with the file in the sandbox
#: you can open files in the standard way just use
#: the self.sandbox.path variable
return """File downloaded, but you will not found it because at the
end of the task the sandbox will be cleaned"""

Following you can see the source code of the protagonist of this post, the sandBox.py file.