Programming Stackless Python

You can import continuation and do other Stackless Python-based programming yourself with only a bit of effort. This article explains how to get started: where to find the files you'll need, how to install them, and how to verify that your installation is working properly.

Clean code

Christian Tismer works almost entirely with WinNT, and he insists that Stackless remains a research project. While he's spent little time polishing Stackless's packaging, his source code is so well-written that it's easy to build your own system, even under Unix and MacOS.

If you're working with Win95, Win98, WinNT, Win2000, or Palm III, you'll simply download the installation packages available through the Stackless Python home page. Unix workers must patch and recompile the Python source code. This means a few extra steps:

The Stackless web site mentions only Solaris as a Unix where Stackless currently works. In my experiments, though, the straightforward recipe above turned up no problems with a variety of compilers used over FreeBSD, Tru64 Unix, and Linux. Conversations with other Stackless users confirm that the source works well on all known Unix varieties.

What do I do with it?

Once you've built your first Stackless Python, the immediate "sanity test" you'll want to run is simply to start the interpreter interactively and

import continuation

Any problem at this stage is likely to mean that you did one of the steps above out of order or that you're somehow executing an older copy of Python than the one you just installed.

For a slightly more interesting test, you can exercise a continuation-based counter. This continuation both receives and returns a parameter:

Remember the summary from "Introduction to Stackless Python," the first article in this series: A continuation represents the future computations of a program. By parameterizing such a future path, looptest codes a simple count-down loop. On each iteration of the loop, the code tests and prints the current value of the parameter k. The invocation of my_continuation that follows then advances the program closer to the end state. This is the "moral equivalent" of looping by way of recursion.

More specifically, my_continuation is the continuation that is ready to branch to the k = my_continuation.update() assignment. When the assignment is executed, k receives the value returned by the continuation. If k is zero, the continuation is discarded and processing passes back outside looptest(). Otherwise, the effect of my_continuation(k - 1) is to transfer control back to the assignment, but with a new value -- one fewer -- for the argument.

Don't let this explanation frighten you. Continuations are so abstract that everyone finds them murky the first time through. The point of Stackless Python, though, is that continuation generalizes recursion and other representations of control-flow state. With continuations, looptest can be implemented
so that its computations are mobile from one interpretive environment to another, perhaps even on a different machine, or can be made persistent and restarted later, or support other concurrency models without threading's bookkeeping difficulties.

Now that you can count with a continuation, you're ready to begin to write your own Stackless-reliant Python applications. In fact, whether you understand continuations or not, simply having the example run on your own machine qualifies you to move on to the concurrency structures, all of which are easier to grasp. Microthreaded work is best done with one of the packages Will Ware, Just van Rossum, and Mike Fletcher have written. Eventually the best aspects of these will be consolidated. Ware maintains pointers to current versions of all the microthreading
alternatives. Tismer's "Continuations and Stackless Python" paper includes examples of generators. Several coroutine definitions based on Stackless are still in revision by Gordon McMillan and others.

The idea is the thing

Tismer emphasizes that it's the idea that's most important about Stackless. Implementation details and even the programming interface are likely to change during the next year, as he continues to strengthen and simplify the Stackless architecture. On the other hand, any work you do now with Stackless is unlikely to be "orphaned." Tismer and the others are careful with their designs. Suppose, for example, you exploit a microthreading definition that's eventually abandoned. The
Stackless gang will probably support one even more general, so you can easily wrap their definition for compatibility with your applications.

This fluidity makes it hard for beginners to gain a clear conception of exactly what Stackless Python is: a fundamental rewrite of Python's implementation to make interesting new concurrency structures far more feasible. The essence of this rewrite has become clear over the first few months of Stackless's evolution. Stackless advocates say it's ready for release now.

At the same time, the Stackless implementation will almost surely continue to develop, at least in its details. Even more confusing is that, as it's currently coded, the only manifestation of Stackless apparent at the level of Python coding is the continuation module. This is confusing because this "presentation" layer is perhaps the most volatile part of Stackless's design. Experiments by Just van Rossum and others have demonstrated that "native" implementations of coroutines can be twice as fast as those built over continuations.

The conclusion: You can work with Stackless now, and make good use of it. Or you can check back next year, when it'll be far better packaged for the "retail customer." While many of the details will change, you should always be able to
recognize the theme of innovative concurrency abstractions. The complexity of the current Stackless story is a "transient" that'll fade away once it becomes more widely accepted and standardized.

The next article, on the "Python roadmap," will relate how we're likely to get from here to there.