Mod_python's PSP: Python Server Pages

The new 3.1 version of mod_python introduces several major additions and
enhancements over the previous 3.0 version. They are PSP, Cookie, and Session
support. This article will introduce the first addition on the list, PSP.

Python Server Pages (PSP), as you probably guessed already, is a way to inline Python in HTML or
XML documents. The server interprets this inlined code to produce the final
HTML sent to the client. This approach has become popular with tools such as
JSP, PHP, and ColdFusion.

Several other projects share the name "PSP", including Webware PSP and Perl
Server Pages. The canonical name of what this article describes is therefore
"Mod_python PSP", but for simplicity I will often refer to it as simply PSP.

PSP Story

Inlining Python in HTML is a subject of some controversy. Some people
consider code inside HTML a bad programming practice, since it utterly violates
the Model-View-Controller paradigm by placing the application logic inside the
presentation layer. (I am inclined to agree with this, but as you will see
later on, there are ways to use PSP that actually comply with the MVC model.)
Bad as it may be, millions of PHP users show a clear demand for this style of
programming — developers are having success implementing sites this
way.

The inclusion of PSP in mod_python was rather unexpected. I have always
maintained that something like this is outside the scope of mod_python, whose
main objective is Apache-Python integration, not high-level web application
tools and frameworks. Meanwhile, expecting "batteries to be included", many new
mod_python users had expressed disappointment on the mailing lists that
mod_python lacked PSP-like functionality. Quite a few frameworks worked with
mod_python, but it seems like to choose a framework, one would have to try them
all out first and make a decision personally.

Then one day an announcement came across the Python mailing list from Sterling Hughes (a PHP core
developer, but obviously a Python fan as well!) regarding mod_psp. Mod_psp was
an Apache module written in C that provided bare-bones Python-in-HTML
functionality in a clean and simple way. Mod_psp was not thread-safe and had
some syntactical short-comings, but it appeared as a great starting point for
something that could be included in mod_python, mainly because it was fast and
written specifically for Apache. Even better, Sterling had agreed to donate
his code to ASF and had spent time on the initial work of integrating mod_psp
into mod_python.

PSP Objectives

Early in the PSP-related discussions on the mod_python development list, I
compiled a list of criteria that I felt were key to a successful
implementation:

There should be no new language to learn. Many other frameworks, mainly as
way to overcome the white space conflict between Python and HTML, introduced
alternative syntax, often complex enough that it appeared as a separate
language somewhat resembling Python. In my opinion this contradicts the spirit
of simplicity and clarity of Python. New syntax can also be a significant
deterrent for developers who rely on language syntax support features of their
editor.

It should be as fast as possible. Mod_python solves many complex and
low-level operating system and networking issues for the sake of performance,
something that most Python developers do not have the skill or time to deal
with. A PSP implementation should comply with this notion by being fast and
clean. There is no value in hacking together yet another slow parser
implemented in Python. The fact that PSP is a flex-generated C scanner adds
real value in that it is not something that most people "can try at home".

It should require no Python semantics in HTML. Python uses indentation to
denote blocks of code. Indentation has no significance in HTML. PSP should not
require HTML to be indented.

It should require no HTML semantics in Python. Since HTML pays no attention
to indentation, it seems there should be another way to denote blocks within
PSP. The first temptation is to introduce a code block delimiter character into
Python. PSP should avoid this.

PSP Syntax

PSP is similar to early JSP, delimiting its code using the
greater/less-than/percent tokens (<% and
%>).

Similarly to JSP, PSP has four types of entities:

Code represents the Python source code that drives the logic of
how the final output is produced. It's enclosed in <% and
%> markers.

An expression is Python code whose resulting string representation
becomes part of the final output. They are enclosed in <%= and
%> markers.

Directives are special instructions to the PSP processor.
Directives are enclosed in <%@ and %>
markers.

Unlike HTML comments, PSP comments are removed by the PSP parser
and never make it into the final output. They are enclosed in
<%-- and --%>.

Admittedly, this syntax is rather primitive and not XML compliant, but it's
a start. Most likely, later versions will feature an alternative XML-compliant
syntax, opening the door for XSLT-generated PSP pages and other XML
goodness.

The thorny issue of indentation is addressed by making the last indentation
in effect "stick" throughout the HTML that follows it, with a recommendation to
use meaningful comments to make code more readable, for example:

<%
if x == y:
# begin
%>
... some html ...
<%
# end
%>

The above code snippet also demonstrates a subtle syntactic difference
introduced by PSP — the indentation of last line of code matters even if
it is a comment.

In the above code, it's easy to forget the block-terminating comment
(# end in this case). Without it, some html becomes
part of the else block and will only be included in the final
output when the user in administrators condition is false.

In general, this syntax works quite well. Its only minor limitation is that
it takes more space (e.g., three lines to terminate a block), though some will
consider it a feature.

Hello World Example

For the impatient, here is a quick Hello World without much explanation. To
use PSP you have to configure Apache and mod_python to use the
mod_python.psp handler. Here is the relevant part of the Apache
config:

req.write("""<html>
""")
import time
req.write("""
<h1>Current time is
""");req.write(str(time.ctime()));req.write("""</h1>
</html>""")

The above Python code is then compiled into a code object using Python's
built-in compile() function. PSP caches that object and reused it
fo subsequent requests, unless the source file changes on disk.

The strange use of semicolons in the resulting Python code is there to try
to preserve line numbering. The PSP parser is not capable of producing any
errors. Bad PSP will simply result in bad Python, which will cause compilation
errors from the Python interpreter. Having the line numbers reported by the
interpreter correspond to the line numbers in the original PSP page helps
tremendously in debugging.

Global Variables

Several variables exist in the global namespace at the PSP page execution
time. These variables, therefore, can be used without assigning a value to them
first. They are:

1. req

req, the mod_python Request object. This means that all of the
advanced functionality of mod_python is still available within the PSP
pages.

2. psp

psp, an instance of PSPInstance, which provides
access to a small PSP-specific API. This object has the following methods:

set_error_page(filename)

This allows you to specify a PSP page to be invoked when a Python error
occurs. This is useful for customizing error output, similar to the
errorPage directive in JSP.

apply_data(object)

This method will call the callable object object, passing form
data as arguments and return the result. If you are familiar with JSP, this
works much like setProperty. If, for example you have an object
defined as follows:

class Car:
def __init__(self, color):
self.color = color
# etc.

Then a PSP page called as result of a form submission (the form contains a
field named color) can do this:

<%
car = psp.apply_data(Car)
%>

This will call the callable object Car (classes are callable),
passing it the value of form field named color, resulting in an
instance of Car which is assigned to car.

redirect(location)

This can be used for redirection from within PSP pages. It's important to
call this function absolutely first in the PSP page, because redirection cannot
happen after there is any output sent to the browser.

3. form

form is the form data in a dictionary-like object (mod_python
FieldStorage). Merely mentioning this in the code causes a
FieldStorage instantiation, thereby consuming all
POST input. PSP uses Python's introspective qualities to determine
whether a piece of code uses the form variable. If the form variable is not
mentioned in the code, then no FieldStorage is instantiated.

4. session

session Similarly, PSP's the behavior changes if you mention
this variable inside the PSP page. When PSP detects that a page refers to the
session variable, it automatically creates a
mod_python.Session object, which will generate session cookies and
turn on session locking ensuring that each unique session can only have one
active request to this page at a time.

Directives

At this point PSP supports only one directive.

<%@ include file='filename'>

The PSP parser will replace this directive by the contents of the file
filename. This can be a very useful feature, although it presently
carries the limitation of complicating debugging because it throws off any
correspondence of original line numbers to the line numbers in the resulting
Python code.

Debugging

As I already mentioned, the PSP parser produces no errors. This may change
in the future, but for now this is the case. Only the Python interpreter
produces errors, but those errors will refer to the PSP-generated Python code,
which is not visible to the developer. It can be difficult at times to
associate the error condition reported by Python with the original PSP
source.

To aid in this, the PSP handler provides the ability to peek at the
intermediate Python code produced by the parser. This only works when
PythonDebug Apache configuration directive is On. If
you append an underscore to the PSP URL, you will receive a nice listing
showing original PSP on the left and the resulting Python code on the
right.

If the original link were http://localhost/test.psp, then
http://localhost/test.psp_ will show the PSP-generated Python
source code. For this to work, you must register the .psp_ extension (with the underscore) with AddHandler:

AddHandler mod_python .psp .psp_

Using PSP as a Templating System

Most *SP's only support the mode of operation where the code is inlined in a
web page referred to in the URL. As I mentioned above, this is not the best
programming practice because it places the application logic inside the
presentation component. Mod_python PSP can also work as a flexible templating
system from within Python code. I advocate this method of using PSP because it
allows for a clean programming style, separating presentation from logic.

Let's look at an example of such usage. Since we're not using PSP as a
handler, we will use the Publisher handler instead. Since the
Publisher handler is outside the scope of this article, I will
instead list the code that I think is self-explanatory. If you need more
information on the Publisher, I recommend looking at the
mod_python tutorial section in the standard documentation.

Here is a snippet of the relevant Apache configuration for using the
Publisher:

By the magic of the Publisher handler, you can invoke the
hello() function in the script via
http://localhost/pubpsp/hello. Pass a name to the
hello() function with
http://localhost/pubpsp?name=joe.

In the above example, we load the PSP template by creating an instance of
the psp.PSP class. At the time of instantiation, PSP either
translates it into Python source code and then compiles it or loads it from the
PSP cache. (Yes, caching is on even when PSP is used this way).

The next line calls the template's run() method, passing it a
vars dictionary. This is when the template is actually executed
and its output is sent to the client. The vars dictionary is a
list of variable names that will be added to the global namespace of the
template just before it is executed. We pass it a variable called
greet which is referred to in an expression inside the
template.

Nested PSP Templates

A nice feature of the PSP templates is that one template can contain
references to another in an expression. Let's expand the above example. The
Apache configuration stays the same, but we'll add another template file called
time.tmpl:

In the above example we instantiated an additional template,
time_tmpl. Note that this time we passed a variable called
now as part of the constructor rather than an argument to the
run() method. PSP allows you to pass variables either or both
ways. Then we pass the time_tmpl PSP object to the
hello_tmpl.

In this example, the time_tmpl is parsed and compiled at
instantiation time, but is executed only when the container template's
run() method is called.

This example is, of course, completely impractical, but it demonstrates a
very useful feature. Imagine that you have a complex site that contains a
dynamically generated menu. You can place such a menu into a template of its
own and include it in the other templates of the site.

Conclusion

The PSP functionality included in mod_python 3.1 is powerful and versatile,
even though it is only the first version. Its ability to be used as a class
from within the Python code makes it suitable for clean web development in
accordance with the MVC paradigm.

Gregory Trubetskoy
is the lead developer of mod_python and a member of the Apache Software Foundation.