6. Setting Up Python WSGI Web Application Servers

CherryPy and Web Application Deployment In Brief

CherryPy as a whole is a minimalist Python web application development framework that is not shipped with too many components out-of-the-box, regardless of you wanting them or not. The framework handles all the core necessities you might need (e.g. sessions, caching, file uploads et al.) and leave the rest - and the choice - of what to use and how to use to be decided by you. It separates itself from other Python frameworks with its simplicity to get online using the shipped, ready to deploy HTTP/1.1-compliant, WSGI thread-pooled Web Server.

Web Application Deployment

In regards to all Python WSGI web applications, deployments consists of preparing a WSGI module that contains a reference to your application object which is then used as a point of entrance by the web-server to pass the requests.

Note: However, in case of using CherryPy's own server, the process becomes simpler and you don't particularly need to worry about it.

In our article, we will see two different ways of application deployment:

Using CherryPy's default web-server, excellent for a majority of applications, and;

WSGI

WSGI in a nutshell is an interface between a web server and the application itself. It exists to ensure a standardized way between various servers and applications (frameworks) to work with each other, allowing interchangeability when necessary (i.e. switching from development to production environment), which is a must-have need nowadays.

Using Nginx As A Reverse-Proxy

Nginx is a very high performant web server / (reverse)-proxy. It has reached its popularity due to being light weight, relatively easy to work with and easy to extend (with add-ons / plug-ins). Thanks to its architecture, it is capable of handling a lot of requests (virtually unlimited), which - depending on your application or website load - could be really hard to tackle using some other, older alternatives.

Remember: "Handling" connections technically means not dropping them and being able to serve them with something. You still need your application and database functioning well in order to have Nginx serve clients responses that are not error messages.

Python (WSGI) Web Application Servers

They operate by accepting a Python module containing - as previously explained - an application callable to contain the web-application and serve it on a network.

Although some of them are highly capable servers that can be used directly, it is recommended to use Nginx in front for the reasons mentioned above (e.g. higher performance). Similarly, development servers that are usually shipped with web application frameworks are not recommended to be used in production due to their lack of functionality - with a few exceptions, of course!

Some Popular Python WSGI web servers are:

CherryPy

Gunicorn

uWSGI

waitress

CherryPy's Application (HTTP) Server In Brief

CherryPy's pure Python web server is a compact solution which comes with the namesake framework. Defined by the [CherryPy] project as a “high-speed, production ready, thread pooled, generic HTTP server,” it is a modularized component which can be used to serve any Python WSGI web application.

CherryPy Web Server's Highlights:

A very compact and simple to use pure-Python solution

Easy to configure, easy to use

Thread-pooled and fast

Allows scaling

Supports SSL

uWSGI

Despite its very confusing naming conventions, uWSGI itself is a vast project with many components, aiming to provide a full software stack for building hosting services. One of these components, the uWSGI server, runs Python WSGI applications. It is capable of using various protocols, including its own uwsgi wire protocol, which is quasi-identical to SCGI. In order to fulfil the understandable demand to use stand-alone HTTP servers in front of application servers, NGINX and Cherokee web servers are modularized to support uWSGI's [own] best performing uwsgi protocol to have direct control over its processes.

uWSGI Highlights

uWSGI comes with a WSGI adapter and it fully supports Python applications running on WSGI.

It links with libpython. It loads the application code on startup and acts like a Python interpreter. It parses the incoming requests and invokes the Python callable.

It comes with direct support for popular NGINX web server (along with Cherokee* and lighttpd).

It is written in C.

Its various components can do much more than running an application, which might be handy for expansion.

Currently (as of late 2013), it is actively developed and has fast release cycles.

It has various engines for running applications (asynchronous and synchronous).

It can mean lower memory footprint to run.

Preparing A Simple CherryPy Application With The "app" Object Exposed

Let's begin our deployment example with creating a new CherryPy application to use as a sample.

Edit “wsgi.py” using nano

The wsgi.py file will be used to expose an entry point to your application. In our example, inside this file we will import the application module (app) and run it directly using CherryPy's WSGI web server, or, pass it on to uWSGI which will contain it and run.

Preparing The System For CherryPy Deployment

Updating The System

In order to have a stable deployment server, it is crucial to keep things up-to-date and well maintained.

To ensure that we have the latest available versions of default applications, we need to update our system.

For Debian Based Systems (i.e. Ubuntu, Debian), run the following:

aptitude update
aptitude -y upgrade

For RHEL Based Systems (i.e. CentOS), run the following:

yum -y update

Setting up Python, pip and virtualenv

Note for CentOS / RHEL Users:

CentOS / RHEL, by default, comes as a very lean server. Its toolset, which is likely to be dated for your needs, is not there to run your applications but to power the server's system tools (e.g. YUM).

In order to prepare your CentOS system, Python needs to be set up (i.e. compiled from the source) and pip* / virtualenv need installing using that interpreter.

On Ubuntu and Debian, a recent version of Python interpreter which you can use comes by default. It leaves us with only a limited number of additional packages to install:

python-dev (development tools),

pip (to manage packages),

virtualenv (to create isolated, virtual

Note: Before continuing with installation of certain applications, especially if building from source, you might need to install essential development tools build-essential using the following command:

aptitude install -y build-essential

python-dev

python-dev is an operating-system level package which contains extended development tools for building Python modules.

virtualenv

It is best to contain a Python application within its own environment together with all of its dependencies. An environment can be best described (in simple terms) as an isolated location (a directory) where everything resides. For this purpose, a tool called virtualenv is used.

Run the following to install virtualenv using pip:

sudo pip install virtualenv

Downloading And Installing CherryPy

CherryPy framework can be installed using the pip package manager.

Run the following to install cherrypy using pip:

# Install CherryPy Framework and HTTP Web-Server
pip install cherrypy

Downloading And Installing uWSGI

It is always the recommended way to contain all application related elements, as much as possible, together inside the virtual environment. So we will download and install uWSGI as such.

If you are not working inside an environment, uWSGI will be installed globally (i.e. available systemwide). This is not recommended – always opt for using virtualenv.

To install uWSGI using pip, run the following:

pip install uwsgi

How To Handle Application Dependencies Using pip

Since it is highly likely that you have started the development process on a local machine, when deploying your application, you will need to make sure that all of its dependencies are installed (inside your *virtual environment).

Creating A List of Application Dependencies

The simplest way to get the dependencies on the production environment is by using pip. With a single command, it is capable of generating all the packages (or dependencies) you have installed (within your activated environment, if not, globally on your system) and again with a single command, it allows you to have them all downloaded and installed.

Note: This section contains information which is to be executed on your local development machine or from wherever you want to generate the list of application dependencies. This file should be placed inside your application directory and uploaded to your server.

Using "pip" to create a list of installed packages:

pip freeze > requirements.txt

This command will create a file called requirements.txt which contains the list of all installed packages. If you run it within a virtualenv, the list will consist of packages installed inside the environment only. Otherwise, all packages, installed globally will be listed.

Downloading From A List of Application Dependencies

Using pip to install packages from a list:

Note: This section contains information which is to be executed on your production (i.e. deployment) machine / environment.

pip install -r requirements.txt

This command will download and install all the listed packages. If you are working within an activated environment, the files will be downloaded there. Otherwise, they will be installed globally - which is not the recommended way for the reasons explained in the previous sections.

Begin Deployment: Download, Install And Set Up Nginx

Regardless of the choice of server, our CherryPy application will go online behind Nginx for the reasons we have mentioned in the previous sections. So, let us download and configure Nginx first and continue with working application servers.

Running and Managing The CherryPy Application Server

To start serving your application, you just need to execute server.py using your Python installation.

Run the following to start the server as configured:

python ~/my_app/wsgi.py

This will run the server on the foreground. If you would like to stop it, press CTRL+C.

To run the server in the background, use the following:

python ~/my_app/wsgi.py &

When you run an application in the background, you will need to use a process manager (e.g. htop) to kill (or stop) it.

Serving Applications Using uWSGI [*]

Albeit being extremely capable and powerful, CherryPy's own HTTP server is not for all set ups or deployments. If you require the ability to tune a lot of options to match your desired configuration settings, uWSGI might be the solution for you.

Running The Server

uWSGI has a lot of options and configurations with many possible ways of using them thanks to its flexibility. Without complicating things from the start, we will begin with working with it as simply as possible and continuing thereon with more advanced methods.

Note: Make sure to be in the my_app folder before executing the below commands as otherwise uwsgi will not be able to find wsgi.py nor import the application object app.

Simple usage example:

uwsgi [option] [option 2] .. -w [wsgi file with app. callable]

To run uWSGI to start serving the application from wsgi.py, run the following:

uwsgi --socket 127.0.0.1:8080 --protocol=http -w wsgi:app

This will run the server on the foreground. If you would like to stop it, press CTRL+C.

To run the server in the background, run the following:

uwsgi --socket 127.0.0.1:8080 --protocol=http -w wsgi:app &

When you run an application in the background, you will need to use a process manager (e.g. htop) to kill (or stop) it. See the section below for more details.

And that's it! After connecting your application server with Nginx, you can now visit it by going to your droplet's IP address using your favourite browser.

http://[your droplet's IP adde.]/
# Hello, world!

Further Reading

If you would like to learn more about Python web-application deployments, you are recommended to check out our following articles on the subject for a better general understanding: