For load balancer I consider HAProxy
For production HTTP sever I consider Nginx
For wsgi servers I mean servers which directly handle wsgi app (gevent, waitress, uwsgi...)
-* means one to many connection
- means one to one connection

There is no static content to serve. So I wonder if production HTTP server is needed.

What are the pros and cons of each solution?

For each scenario (1-3), in place of wsgi server is there any advantage of using wsgi container server (uWSGI, gunicorn) rather then raw wsgi server (gevent, tornado..)?

I'm also wondering which solution is the best for websockets or long polling request?

2 Answers
2

1: You almost certainly want an HTTP reverse proxy like nginx to deal with "spoon feeding" slow or stupid client programs; some of your users will be on slow connections; the reverse proxy usually can wait for the request to be fully received before contacting the application, then slurp the full response from the application quickly (so that it can move on to other requests), and then feed that back to the client as slowly as they require. If you're using a reverse proxy anyways, there's not much reason to consider a tcp level loadbalancer, too; since the reverse proxy can already solve that problem. This is especially true since tcp load balancers aren't application aware, and can't skip upstream hosts that are "reachable" but "sick", they will happily proxy for servers that are returning "500 Internal Server Error" responses to health check requests. they're usually needed only at the extreme edges of your network at very high load.

2: which application container is right for you depends as much on the application as it does on the shape of your workload; to take advantage of async containers like tornado, your application must be written in a special way; and can't use all of the nice/convenient frameworks that are available for wsgi in general; on the other hand, you will need them for some features like long polling and especially websocket, these features are not practical (or even possible) in things like uwsgi.

but, not all containers are created equal; many speak only HTTP, which is not a CPU friendly protocol, containers like uwsgi are designed to optimize the http parsing work so that only the reverse proxy has to do it, from there out, easily parsed binary protocols are passed from one process to the next.

3: websocket is still very new, and support in python is sparse. The most mature options seem to be the implementations available in tornado and in twisted; Neither can be hosted in uwsgi, and cannot be proxied behind nginx. there are other reverse proxies that can handle websocket, though, for example, varnish.

Ad. 2. I understand the difference between sync and async programming. The question was about using containers (like gunicorn + tornado workers ) rather then solo wsgi servers (tornado)
–
Robert ZarembaJan 5 '13 at 1:48

the solo wsgi servers are, without exception, not useful for any sort of async processing; you can't use them for websocket and they are unlikely to be a good fit for long polling.
–
SingleNegationEliminationJan 5 '13 at 2:24

Sorry for my misleading, but I always consider some load balancer server (nginx / haproxy) for every scenario. So the solo wsgi server isn't alone. So the question is: is it worth to have eg: nginx -* gunicorn + tornado worker, instead of nginx -* tornado. Thanks for your engagement!
–
Robert ZarembaJan 5 '13 at 3:29

neither nginx nor uwsgi nor both will help with websocket! to put a websocket service behind the same port as either, a secondary reverse proxy is needed, one that can distinguish traffic for the websocket application from the http/1.1 traffic. Varnish can do this, but it's not quite as versitile in other ways as nginx: it can't speak fastcgi/scgi/uwsgi and it can't serve static files: You probably need all of Varnish, Nginx, Tornado, and uWSGI.
–
SingleNegationEliminationJan 5 '13 at 3:35