A blog about modern web development, programming in python, django, flask, javascript, coffeescript and other

Sunday, July 8, 2012

Unstable HTTP services: what we can do to easily handle that?

The story: I have several HTTP service providers which works quite unstable. Yes, I had to have in mind this during development. But we're all thought that issues are "temporary"and will gone when we going to production. We accurately added logging.error in every place and move on with other stuff.

But our expectation about temporary nature of service behavior will never happen. Service sometimes work slowly, sometimes return HTTP errors and so on. We receive tons of exceptions every day. We had to do something with that.

The solution: Here is safe_exec decorator which help solve this problem. You can specify how many times you want to try execute function, what's timeout between them and what exceptions are expected during execution decorated function. For example, urllib2.urlopen may generate urllib2.URLError or urllib2.HTTPError.