Thanks to shlomif for the link to Neo-Tech, a hysterically funny cult. I highly recommend reading through the site (think of a cross between Heinlein's opinions as interpreted by a 16 year old, traditional rich right-wing American's political and economic agenda, and plain endearing wackiness.)

They call Bush a "closet secular humanist"! They explain how "The Zon/string-theory speculates that by orchestrating multi-dimensional quantum vibrations or gravity-unit ripples, technologically-advanced people could not only birth universes, but could design their initial conditions for optimum efficiency in developing invaluable conscious life yielding maximum returns on investment." They call Libertarians "left wing". They even explain what those mysterious Illuminati are up to.

Two weeks where I used computer a lot less, due to pain in wrists and hands. It's still not completely cleared up. I did feel well enough to finally release Cog version 0.3.

Cog is the Checkpointed Object Graph object database, providing semi-transparent persistence for large sets of interrelated Python objects. It handles automatic loading of objects on reference, and saving of modified objects back to disk. Reference counting is used to automatically remove no longer referenced objects from storage, and objects will be automatically be attached to the database if a persistent object references them.

It was originally written by Oliver Jowett, but I am now maintaining it.

I just got back from the week long Zope3 coding sprintathon in Rotterdam. Lots of fun, very educational, and now I have even more obligations of code I need to write for free software projects :)

The sprint was an impressive commentary on the success of free software as a commercial enterprise, with around 25-30 developers from 15 different organizations. Unlike Zope2, which was in general developed by a single company - Zope Corporation - Zope3 is being developed in a series of coding sprints in addition to the individual ongoing contribution of developers. The sprints and development are done, in general, by programmers at companies whose business is based on Zope2.

There are 95 companies listed as Zope Solution Proviers on zope.org, but in practice there are probably many more, as well as even more companies who sites run on Zope. No doubt almost all of them are interested in Zope3, and quite a few are putting in some resources (e.g. giving workers leave for a few days to go to a sprint) to make it happen.

On the technical side, it's interesting to compare Twisted to Zope3. While they have different purposes - Twisted is a general purpose networking framework and libraries, Zope3 is a web application server, they are both fairly large Python frameworks.

Someone called the Twisted developers "abstraction astronauts", but compared to Zope3 Twisted is so concrete you could build bomb shelters with it. This has a price of course, Zope3 is quite slow and only does 1 request a second on my laptop, but it has not been optimized at all. Knowing the people involved I don't doubt their claims that they have clear paths for optimization, and I'm sure the final release will have decent speed.

In any case, the abstraction is a good thing. Zope3 is an inherently complex application, and the new design is quite superior to the kludge known as Zope2. And Zope2 is the best content-management oriented web framework I know of (the next person to point to their favorite PHP project gets a $10 voucher for Toys R Us where they can buy some more toys.)

Twisted has been running applications for most of its existence. For example, http://twistedmatrix.com has been running using Twisted's web server for about 2 years. Since this also includes user homepages and the #python wiki (MoinMoin CGI), this provides some real world testing under real conditions. In addition, a number of commercial and open source projects are using Twisted, some of which have already been deployed. Zope3 on the other hand has not been used for any real world usage, other than developing itself - not even a test server for the development project. This will soon be changing as a couple of projects will probably start using it once the alpha is out.

I am confident in the Zope3 source code quality however, beyond the design itself. The use of interfaces means there's a clean separation between the requirements and the implementation, which does of course suck in some places. In addition, there are a *lot* of tests. Twisted has 75,000 lines of code, Zope3 has 150,000. Twisted has 350 automated tests and about 15 tests that need to be checked by users. Zope3 has 2700 automated tests! This will mean refactoring to deal with real world requirements will be much easier.

All in all, I'm really looking forward to Zope3. I'd probably be jumping up and down going "GIMME GIMME GIMME I WANT IT NOW" except I'm trying to get out of doing content management website jobs.

Spending my time getting the Cog object database up to speed for use in a project I'm working on. I've gotten it to the point that developers only need to mark their classes as persistent and then mark them as dirty when necessary, and it Does The Right Thing.

Next task is rewriting the storage backend, probably using the Python bindings to bsddb. Long term features I want to add are a backup script, GC script, hooks for indexing stored objects, maybe rollback support, but I'll do those after I finish the rest of the contract.

As usual in Python, you need to work a lot less than you might expect - Python's pickle library already has all the hooks necessary for serialization to an object database. So Cog can concentrate on doing all the other parts. It's quite a nice package even as is, other than a few rough spots.

I added multicast support to Twisted. Or to be more accurate, the tests for the multicast code pass - I have no idea if it works in the real world, and probably won't until we get a Real User to use the code.

Together with the rewrite of the old UDP API, we now support (in theory :) another small corner of the networking world.

I realized the issue isn't just with exceptions - it can be with any operation (e.g. log rotation on a signal might be interrupting a log write method, and who knows what happens then).

Yet another benefit of event loops is that the problem is easier to solve. Instead of raising an exception (or rotating the log or whatever), you do this by registering a method to run in the next iteration of the event loop. Because you are forced to split up your tasks into small, self-contained, short running event handlers anyway, your program by design will be much easier to structure so that this is not a problem. So, instead of rotating log on signal, you would tell event loop to run log rotation method in its next iteration.

I'm going to have to change Twisted's rotate-log-on-signal code and possibly other signal handlers so they work that way, if they don't already.

Software Methodology

My father wrote a paper on software methodologies which might be of interest to people here, especially if you're interesting in agile programming and the like.

On Saturday I was at the Lightweight Languages Workshop at MIT, and here are notes about some of the talks. There's a webcast available from the website.

Erlang

First talk was Erlang, which is very interesting. It's a functional language,
and designed for concurrency - you can launch tens of thousands of processes,
and pass messages between them. And this is done fast - 2 microseconds to
start a process. Apparently Oz shares a lot of the concepts here as well.

OS-level services in Scheme

Second talk was about the ways PLT Scheme (MzScheme?) is incorporating OS-level
functionality into the language.

PLT Scheme OS-like capabilities that I found interesting:

killing threads cleanly

Indepent GUI eventspaces (e.g. independent regarding modal dialogs).

custodian manages resources for threads so you can easily shut it down.

enforce abstractions (like Java's private?) and then add hooks for
bypassing it for debugging.

This is very useful for IDEs. They have example IDE that lets you run say,
infinite loop in interpreter, without the GUI freezing up, and because you can kill threads you can
cancel this operation. In addition, GUI you
program via IDE runs in its own event loop so it doesn't interfere
with IDE's event loop.

Apparently the author is spending a lot of time making the blocking OS
level facilities not block, so he can do this sort of thing.

Asynchronous Exceptions in Python

The fourth talk was about raising exceptions in signal handlers in
Python, and the problem this causes. Basically the issue is that
in e.g. this code:

f = open("file")
try:
s = f.read()
finally:
f.close()

the file may never be closed, since a signal might raise an exception
right after file is opened but before try/finally. The obvious solution
is too disable the signal around this code, but this is TOO MUCH TYPING,
and you MIGHT FORGET to reenable the signal. So what was suggested?
Adding new keywords! They gave following example code:

block:
f = open("f")
try:
unblock:
s = f.read()
finally:
f.close()

where block/unblock do that to signals. I think this is terribly wrong.
First of all, their own example was wrong, it should actually have been:

This is totally ridiculous - you need to wrap each block of code this way,
it blocks all signals instead of just the appropriate one, and the syntax
totally violates the standard behavior of existing keyword pairs in python,
since they are on different indentation levels.

I think raising exceptions from signal handlers is just a messy idiom,
and they just shouldn't be doing it. And given you don't need a keyword
to solve the problem, you just can use the signal module, their suggestion
is just badly implemented syntactic sugar, with side effects that extend
beyond the problem they want to solve.