I've got a python script that I want to run via crontab. My crontab looks like this:

5,20,35,50 * * * * /var/www/django-apps/callreport/util.py

That script is set up to parse a bunch of flat files and stick the info into a MySQL db, then delete the files. It runs fine from the command line, data is copied to the db, and the flat files are removed. But nothing happens when set to run as a cron job.

In the past, I'd get a mail message when a cron job failed, but I'm not getting any feedback with this one, and I'm still feeling my way through being a sysadmin on this box. What am I doing wrong?

11 Answers
11

The usual problem with 'cron' jobs is that they have zero environment - unlike 'at' jobs which do copy your environment. When something works from the command line and not from 'cron', my experience is that 'environment' is one of the most common problems. Occasionally you run into another problem - 'cron' jobs are not run with a terminal, and occasionally programs get stroppy about this. However, this is in the 1% range compared with 99% for environment issues.

The other key technique I use is to always run a shell script from 'cron'; the shell script ensures that the environment is set correctly and then runs the real program. If a 'cron' job is giving me problems, I can then tweak the script to do useful things like this:

This redirects all the output - standard output and standard error - to a file name. You can build timestamps into the filename if you prefer that to the process ID. Analyzing the log files often reveals the problems swiftly.

The problem was the environment. Part of my script was relying on a path relative to the current working directory. When run as a standalone script from the command line, this wasn't a problem, but cron's current working dir was my home directory (this was my user crontab). Fixing the script to work out the right path solved my problem.
–
saturdayplaceOct 27 '09 at 19:00

When cronjobs that work on the commandline fail to run as expected, it's usually an environment problem for me -- remember that the cronjob will not be running as an interactive shell.

From your commandline, run env(1) and copy these down somewhere. Next, modify your cronjob to run env so you can compare the values. Cron should email you the output (you said it was before); HD mentioned how to configure this.

Check your system logs to see if the cron daemon is actually launching the script

If your jobs are not being logged, try increasing the logging level so that you that cron logs the start of each job (see man cron). For example, if you are running the Vixie cron daemon, this can be done by specifying the -L 1 option (or -L 2 if you also want to check when your job finishes).

From your Python script, as soon as it starts, create an empty file on the /tmp directory (this is another way to help you verify that the script is being called by cron and it's starting without problems)

To receive a mail from cron. Also you can redirect the output to a temporary fail as Dinesh Manne says to check what happened to your command.
Just asking: are you sure that you're using the user which is running the command is the same as the owner of the crontab you're editing?

Force it to crash. Put something wrong instead of #!/usr/bin/python and verify if you get an error report by e-mail - you should set the MAILTO variable to your e-mail address. But check first if the scripts really fails if you ran it from command line.
If you still don't get an e-mail error verify if some e-mails are stucked in the local mail queue (mailq). Maybe this will help figuring out about the silent failure.

Also, on what internal systems does this script depends ? DNS, ldap/nis ? Does it have any hardcoded IP addresses or hostnames that no longer exists (i.e. mysql host) ? Can you correlate the last time it succefully ran with the last change done to the script ? Did someone did a stealth upgrade to python on your machine ? Do you have sufficient disk space and indoes (df -i ) ?

Do you have an empty line at the end of you crontab file ? If you force the script to run as root (bad, use it only for testing) does it work ?

Do you have this job in a user's crontab (Can you see it with crontab -l?) or is it in the system crontab (/etc/crontab)? The system crontab (On linux at least, don't know about other systems) requires an additional user argument that the user crontabs do not.

It sounds like you've already checked, but double check that the script is being run as the correct user. When I troubleshoot cron jobs, I like to use (on my mac) system calls using the say command so that I hear the parameters I'm interested in. So I might add to util.py the following:

import os
os.system('whoami')

Other than that, I'd put my money on a permission issue; your cron job may not be able to do the things you ask because permissions aren't letting it.