Retrieving Mails with Twisted

Programming a POP3 client to retrieve mails in the Twisted framework is more
complex and takes more code. The logic to retrieve mails is all in a class that
subclasses POPClient (from twisted.mail.pop3client).
Due to the event-driven nature of Twisted, it's easier to define a method for
each step, register the success, and failure callbacks to the method to enter
the next step and handle any error, respectively.

In the class MyPOP3Client, the first step to get mails is the
serverGreeting method, which Twisted will invoke when the client
starts. This method invokes the superclass's serverGreeting, and
then logs in to the POP3 server with a user name and password. The
login method returns a Deferred object, invoking the
addCallbacks method to register the do_stat method
(called upon successful login), and the errorHandler method
(called on login error).

Similarly, the do_stat method invokes POP3Client's
stat method to perform a POP3 STAT command, and registers the next
step as do_retrieve. Because the call to method stat
is asynchronous, it cannot return its results to the caller with return values.
Instead, it passes the results as arguments to the success callback registered
to the stat method. The second parameter to the
do_retrieve method is a list, of which the first element is the
number of messages in the POP3 account.

If there is no message in the mailbox, the code calls
reactor.stop to tell Twisted to shutdown. Otherwise, it invokes
retrieve(0) to get the first message. Its success callback,
do_retrieve_msg, handles the message by displaying its summary,
and then retrieves the next message. Because the method
do_retrieve_msg gets invoked for all subsequent messages, the code
uses an instance variable, cur_message, to keep track of the
current message number and to determine when it has handled all messages. When
it has processed everything, it stops the Twisted main loop.

Because the logic of mail retrieval is similar to deletion, both features
are in the same class, MyPOP3Client. The instance variable
deletion denotes the current mode of working. You can see its
initialization in __init__, along with the user name and password.
More interesting is the setting of allowInsecureLogin to true,
which allows login to a server without authentication challenge non-encrypted
transport.

To delete a mail, call the delete method of class
POP3Client. Similar to the core poplib module, this
method just marks the mail for deletion, and the actual deletion occurs when
you send the POP3 command QUIT to the server, as with the
quit method. Finally, Twisted's execution thread stops when the
quitting action completes.

With the implementation of the desired mail handling in class
MyPOP3Client, you can launch the client. With its descriptive
name, the class ClientCreator (from
twisted.internet.protocol) provides a convenient way to start a
communication client. This code passes the reactor and the
MyPOP3Client class to create a ClientCreator, and
begins the mail retrieval by calling connectTCP with the specified
server and port number. Twisted's execution loop then kicks off by
reactor.run().

Invoking do_retrieve_msg repeatedly for all messages is
conceptually tedious and lengthy, when compared to the
DeferredList mechanism which keeps track of several actions and
gets notified when all actions complete, as in the case of sending mails.
However, collecting the Deferreds of multiple calls to
retrieve of POP3Client in a DeferredList
simply does not work in Twisted (Versions 2.2.0 and 2.4.0). The success
callback never gets invoked (see mail-twisted.py).

Doing Telnet with Twisted

Twisted can power a Telnet client in a way similar to, but simpler than, the
POP3 client. the Telnet conversation logic goes in a subclass of
Telnet (from twisted.conch.telnet).

The class MyTelnet overrides two methods of class
Telnet. The first method, dataReceived, is called
when data arrives at the client. It checks the data received and calls the
_write method to send the user name, password, or the shutdown
command accordingly to the server. The second method is
connectionLost, which Twisted calls when the server closes the
telnet session. In that case, the program simply terminates the Twisted
execution loop. The Telnet client starts by using the ClientClient
class, connected to port 4555 of the James mail server.

When to Be Twisted?

The two functionally equivalent programs, one using Python core modules and
the other using the Twisted framework, significantly differ from each other in
terms of programming style and the amount of code. Then when should you use either
of the two options?

For basic programs such as the command-line client of this example, the
Python core networking modules are more desirable due to the simplicity and
performance advantages. However, most real-world networking programs are very
complex, and Twisted's asynchronous programming model is more effective. For
example, BitTorrent, the popular peer-to-peer file sharing client that
performs massive parallel downloading of data chunks from different sources,
uses Twisted. Twisted also works well in programs with graphical user
interface (GUI), because its asynchronous nature fits more seamlessly with the
event-driven programming models of modern GUI frameworks. In fact, Twisted has
integration with popular GUI frameworks including PyGTK, Qt, Tkinter, WxPython,
and Win32.

The other area where Twisted shines is in server programming. A typical
network server uses multithreading so that it can handle multiple clients
concurrently. The asynchronous mechanism of Twisted alleviates the creation and
handling of threads by server programs. In addition, Twisted provides several
protocols on which to build new networking services, enabling rapid development
of complex servers. One such project is Quotient, which adopts Twisted to build
a multiprotocol messaging server that supports a variety of protocols and
services including SMTP, POP3, IMAP, webmail, and SIP.

Kendrew Lau
is a consultant in Hong Kong, with focus on Java, Linux, and other OSS technologies.