Flask, like Django, Pyramid and others, are based on WSGI, an API standard for connecting Python Web frameworks to Web servers. Which is great. However, WSGI is a (fundamentally) synchronous and blocking API.

WSGI was designed for the old fashioned, threaded/pre-forked Web servers (think Apache). It does not blend well with the new breed of asynchronous networking frameworks like Twisted, asyncio or Tornado.

Why is that limiting?

A (single thread/process) of a WSGI based Web server can only handle one request at a time. The code serving a request will run to its end before another request can be served.

Which means if you want to do e.g. something network related within your request handler, like calling out to a Web service, this will again block the incoming request, and in fact the complete thread/process until the outgoing request comes back.

With more and more requests being blocked, this can quickly lead to a situation where all worker threads/processes are blocked, and no incoming request is served at all anymore. Bad, bad.

Please note that this not only applies to doing Web service requests in your request handlers, but any kind of I/O operation which might take longer, like file or database access.

Can we fix it?

Yes, we can ;) We first show an example using Flask with synchronous code. Then we show how to do the same with asynchronous code using Twisted Klein.

This app will respond to HTTP/POST requests sent to /square/submit, extract the form variable x from the request, square that number and return a text response.

E.g. you can test it with

Now, what if you want to use a Web service to actually compute the square?

Please note, this is an example - normally you wouldn't call out to a Web service to compute a square ;) In fact, we do a simple HTTP/GET and not a full flavored Ajax whatever Web service request. It's just an example.

The requests.get method simulates calling out to some Web service. This call will block until the outgoing HTTP request comes back. And hence, the client's original request to /square/submit will also block. And that means: the whole thread/process will block and cannot do more work like serving other incoming requests while still waiting on outgoing requests.

This can quickly get problematic: when calling out to the Web service takes some time, more and more requests will block until all threads or processes of the WSGI container running the app are blocked.

At that point, no new client requests will be served at all!

Note: It's quite common for WSGI containers to run something like 10 worker threads or processes. That means the app above will stall as soon as 10 incoming requests are served that each still wait on their outgoing request to return. And 10 isn't a large number ;)

Alright. Nothing spectacular. It looks almost identical to the Flask version. The most visible difference is that our request handler square_submit() now takes an request arguments provided by Klein, whereas Flask has request as a global object.

Now, the more interesting second Flask example looks like this when converted to Klein (don't worry, I'll go through the details below):

We are doing an (asynchronous) outgoing HTTP request on line 15. For this, we are using treq, a Twisted port of Requests, a HTTP client library that is easy to use.

The request treq.get() will not return immediately (since we are doing a network request), but it won't block (since we are in asynch land).

It will return a Twisted Deferred (elsewhere known as Futures). The Deferred stands for a result of an asynchronous operation, where the actual return value of the operation will only be available later.

To simplify the look and feel of the code, we are using yield, inlineCallbacks and returnValue following a specific variant of asynchronous programming ("co-routines"). This makes the code "look" synchronous, but nevertheless execute asynchronous under the hood.

Once the HTTP request returns (actually, the HTTP response header), the result of treq.get() becomes available in our variable r, which will hold a response object.

To actually retrieve the response body of our request, we now yield from r.content() (line 16).

There are only two other noteworthy details here: we need to decorate the function with inlineCallbacks to make the co-routine style available (line 9), and we cannot return directly, but use the returnValue wrapper (line 20).

Note: The reason returnValue is necessary goes deep into implementation details of Twisted and Python. In short: co-routines in Python 2 with Twisted are simulated using exceptions. Only Python 3.3+ has gotten native support for co-routines using the new yield from statement.

What have we gained?

Above variant using Twisted Klein will sustain a very high number of concurrent requests. It won't block. It will do so even on one thread, running on one CPU core! We finally are able to use asynchronous code inside Web request handlers. Awesome!

What's the downside?

The downside (obviously) is that you need to write your Web code on top of Twisted Klein, not Flask. At surface, Twisted Klein is very similar to Flask - however it is not a fully compatible drop-in replacement.

Flask is a great Web framework, and Twisted Klein makes the fun available in fully asynchronous Web applications. Going asynchronous allows you to do powerful things inside your request handlers - without running into performance problems due to blocking.

Search this Site

Stay Informed

Crossbar.io and Crossbar.io Fabric

Crossbar.io is an open-source project created and managed by the people behind Crossbar.io Fabric.
Crossbar.io provides the application message routing for real-time Web and IoT applications.
Crossbar.io Fabric will support you in managing your Crossbar.io instances.