Writing Servers

Overview

Twisted is a framework designed to be very flexible and let
you write powerful servers. The cost of this flexibility is a
few layers in the way to writing your server.

This document describes the
Protocol
layer, where you
implement protocol parsing and handling. If you are implementing
an application then you should read this document second, after
first reading the top level overview of how to begin writing your
Twisted application, in Writing Plug-Ins
for Twisted. This document is only relevant to TCP, SSL and
Unix socket servers, there is a separate document
for UDP.

Your protocol handling class will usually subclass twisted.internet.protocol.Protocol. Most
protocol handlers inherit either from this class or from one of
its convenience children. An instance of the protocol class
might be instantiated per-connection, on demand, and might go
away when the connection is finished. This means that
persistent configuration is not saved in the
Protocol.

The persistent configuration is kept in a Factory class,
which usually inherits from twisted.internet.protocol.Factory. The
default factory class just instantiates each Protocol, and then
sets on it an attribute called factory which
points to itself. This lets every Protocol access,
and possibly modify, the persistent configuration.

It is usually useful to be able to offer the same service on
multiple ports or network addresses. This is why the Factory
does not listen to connections, and in fact does not
know anything about the network. See twisted.internet.interfaces.IReactorTCP.listenTCP,
and the other IReactor*.listen* APIs for more
information.

This document will explain each step of the way.

Protocols

As mentioned above, this, along with auxiliary classes and
functions, is where most of the code is. A Twisted protocol
handles data in an asynchronous manner. What this means is that
the protocol never waits for an event, but rather responds to
events as they arrive from the network.

This protocol responds to the initial connection with a well
known quote, and then terminates the connection.

The connectionMade event is usually where set up of the
connection object happens, as well as any initial greetings (as
in the QOTD protocol above, which is actually based on RFC
865). The connectionLost event is where tearing down of any
connection-specific objects is done. Here is an example:

Here connectionMade and
connectionLost cooperate to keep a count of the
active protocols in the factory. connectionMade
immediately closes the connection if there are too many active
protocols.

Using the Protocol

In this section, I will explain how to run a server which uses your
Protocol. (In order to see how you should write a
production-grade Twisted server, though, you should read the Writing Plug-Ins for Twisted HOWTO as well).

In this example, I create a protocol Factory. I want to tell this
factory that its job is to build QOTD protocol instances, so I set its
protocol attribute to the QOTD class. Then, I want to listen
on a TCP port, so I make a TCP4ServerEndpoint to identify the
port that I want to bind to, and then pass the factory I just created to
its listen
method.

Because this is a short example, nothing else has yet started up the
Twisted reactor. endpoint.listen tells the reactor to handle
connections to the endpoint's address using a particular protocol, but the
reactor needs to be running in order for it to do anything.
reactor.run() starts the reactor and then waits forever for
connections to arrive on the port you've specified.

You can stop the reactor by hitting Control-C in a terminal or calling
reactor.stop.

Helper Protocols

Many protocols build upon similar lower-level abstraction.
The most popular in internet protocols is being line-based.
Lines are usually terminated with a CR-LF combinations.

However, quite a few protocols are mixed - they have
line-based sections and then raw data sections. Examples
include HTTP/1.1 and the Freenet protocol.

For those cases, there is the LineReceiver
protocol. This protocol dispatches to two different event
handlers - lineReceived and
rawDataReceived. By default, only
lineReceived will be called, once for each line.
However, if setRawMode is called, the protocol
will call rawDataReceived until
setLineMode is called, which returns it to using
lineReceived.

Several other, less popular, helpers exist, such as a
netstring based protocol and a prefixed-message-length
protocol.

State Machines

Many Twisted protocol handlers need to write a state machine
to record the state they are at. Here are some pieces of advice
which help to write state machines:

Don't write big state machines. Prefer to write a state
machine which deals with one level of abstraction at a
time.

Use Python's dynamicity to create open ended state
machines. See, for example, the code for the SMTP
client.

Don't mix application-specific code with Protocol
handling code. When the protocol handler has to make an
application-specific call, keep it as a method call.

Factories

As mentioned before, usually the class twisted.internet.protocol.Factory works,
and there is no need to subclass it. However, sometimes there
can be factory-specific configuration of the protocols, or
other considerations. In those cases, there is a need to
subclass Factory.

For a factory which simply instantiates instances of a
specific protocol class, simply instantiate
Factory, and sets its protocol attribute:

A Factory has two methods to perform application-specific
building up and tearing down (since a Factory is frequently
persisted, it is often not appropriate to do them in __init__
or __del__, and would frequently be too early or too late).

Here is an example of a factory which allows its Protocols
to write to a special log-file:

listenTCP is
the method which connects a Factory to the network.
It uses the reactor interface, which lets many different loops handle
the networking code, without modifying end-user code, like this.
As mentioned above, if you want to write your code to be a production-grade
Twisted server, and not a mere 20-line hack, you will want to
use the Application object.