Sat, 13 Nov 2010

GIVE CODE TO WORLD

An Essay on the State of Interactive Fiction and Free Software

...we want to establish the idea that a computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute

My Dilemma

When I was a child my parents believed computers to be philosophically superior to game consoles, and bought the family a Commodore 128. They were librarians, and as such were great lovers of books and literature. They encouraged me to participate in Banned Book Week and they gave me free rein to use their charge account at the University of Washington bookstore. However firm they stood on any of the sorts of things they denied me, they always wanted me to have access to the written word.

When I asked for a Nintendo they categorically refused, [1] opting instead to give me text adventures. I'd beg for Gauntlet and they'd buy me Planetfall. I wheedled for Metroid, but got A Mind Forever Voyaging. I demanded Zaxxon, but found The Hitchhiker's Guide To The Galaxy beneath the Christmas tree. This was the one sort of game that really appealed to their sensibilities.

In the end I don't really much remember the time spent as a child on neighbors' Nintendos, but I have fond memories of playing Mindwheel with my father late into the night. I used to keep the printer spitting out transcripts of my games to read over lunch breaks at school. I came to love these games so profoundly that when I registered a vanity domain for myself in the mid-90s I chose zork.net, stunned that it was still available.

So here I am, someone with a strong personal connection to Interactive Fiction, someone with the technical ability and desire to create new works, and yet I have stood aside as a conscientious objector to the new IF movement for a decade and a half. When I mention this to people in either the modern IF community or the Free Software community, I get puzzled reactions. After all, didn't IF go through a progression from retail software to hobbyist community? Aren't the biggest users of plain text terminals the Free Software Unix brigade? Shouldn't there be no conflict here?

What I hope to do in this essay is explain my position and describe the forces and factors behind the current state of IF software. I will attempt to describe the biases and positions of people I have never met, so that I can show their positions from my own perspective and perhaps show "my people" in the Free Software community why certain decisions seem to have been made.

For some reason they bought my younger brother all manner of console system. Go figure.

A Little History

I am going to dispense with the boilerplate here and assume that you have either purchased or downloaded the excellent Get Lamp documentary. With that bit of nostalgia-something in your eye, let me boil down the story of the 1990s IF renaissance to the bits most relevant to my dilemma.

There are two pieces of software most relevant to the IF author in 2010: Inform and TADS. [2]

TADS

TADS was a shareware system written in the 1980s, at the height of the retail home computer software market for this sort of thing. It has often been characterized as being aimed more toward people who are used to programming, and is praised for its brevity. It has a longer history of hobbyist games by virtue of being the older of the two major authorship systems, but many of these games pre-date the formation of a cohesive community.

TADS was released as "freeware" in the 1990s, and there have been irregular publications of full source code. The tads3 source distribution's LICENSE.TXT file permits redistribution, restricts publication of modified versions to changes required for "porting" the software, and forbids commercial use of the sources.

The restrictions on modification and fields of endeavor disqualify it under The Open Source Definition or The Debian Free Software Guidelines, keeping it from being distributed as part of a free operating system. Many people accept the license and write great IF with it, but I cannot in good conscience join them.

Inform

Inform is an entirely different sort of beast from TADS, and it has a far more dramatic history. In the early 1990s, Graham Nelson set about reverse-engineering the virtual machine that Infocom used to make porting its games simpler. He also produced a compiler that could produce binaries for this Z-machine, and included some libraries that were useful for writing IF for this VM.

The fact that Inform compiles to the same format that Infocom used gave it a certain cachet, and a community gelled around the software and the language. While TADS developers tended to be programmers wanting to make games, Inform developers often included more of a sort of starry-eyed 1990s medialab type. [3] These folks saw IF as a way to develop new interactive forms of literature and create art. They were authors looking to add interactivity to their creations.

The division isn't quite as clear-cut as the previous paragraph might imply, but the current community of avant-garde IF authors came into its own as Inform developed. Many of the titans of the genre these days use Inform to the exclusion of other systems.

The licensing for this system is a far more complicated question. The license under which most versions of Inform were released had similar constraints to that of TADS: modified versions could not be redistributed if they were "substantially different" from the official version, and "profit" was forbidden. [4]

Since Inform was such a community effort, the libraries of functions that might compile into a final story file were under a real mess of permissions. Some had no license, and thus it would behoove any user of these libraries to request specific permission to publish any story that uses them. Some were under "public domain" licenses, occasionally with explanations of what permissions this included for the benefit of countries that are not the United States.

The libraries remain a sticky problem: you compile them into your own stories, making your resulting IF a derivative work. If the licenses for the libraries do not permit modified works or commercial use, then you need to explicitly request permission for the right to either of these for your story. The core libraries written by Nelson had an exception to allow users to sell their own games, but not all library authors followed this example.

This situation remained up until Inform 6. Things changed when Inform 7 hit the net. Inform 7 is a radical departure for Inform in many ways. It takes the view of IF as fiction first and foremost, and drives it to its logical conclusion: stories are written in a declarative rules-driven syntax based on modern English. It has the potential to introduce IF authorship to a wider audience than ever would have been possible before.

Unlike all previous versions of Inform, Nelson did not publish his sources as he went. To date, most of v7 is unavailable in source code form, and one downloads binaries built by members of the core development team.

For those who don't remember, the MIT Medialab in the 1990s (and to be fair, always) had a lot of optimistic projects involving ways to make text more interactive. The 90s saw a lot of interest in hypertext, the potential of CD-ROMs, and other approaches that don't get as much excitement today. I chose this particular institution in this particular era because the way it was portrayed in Wired at that time seems to capture something about the Inform community.

This language always struck me as astonishingly sloppy. What if one profited from it intellectually? Could I sell copies of Inform to lose money as a tax shelter?

The Seeds of Confusion

Here is where the story takes a confusing turn, because pieces of Inform 7 are source-available. What's more, the source is released to the net under the terms of the Artistic License 2.0, which is both a DFSG-free and Open Source Approved license. Cementing confusion, the pieces that are truly free software include the Inform 6N compiler and the Inform 6 templates.

This leads to the circumstance that inspired this essay in the first place. To any casual observer, it almost appears that Inform 6 has been released under DFSG-free terms and that Inform 7 is hot on its heels. I have been told numerous times that Inform is free software and that I was spreading misinformation.

The troublesome fact here is that the I6N pieces of Inform 7 don't appear to constitute a complete Inform 6 platform. Yes there is a compiler, but the "templates" appear to have been re-tooled dramatically from their I6 Library origins in order to support the output of ni. Seeing this, it's tempting to ignore the Z-machine cachet, pick your favorite language and start from there. [5]

And indeed, why haven't I done this? For starters the Z-machine is the delivery mechanism for IF. Secondly I am the author of the AWK entry in the Cloak of Darkness project, and have dug through enough prior art to realize what an enormous task that would be. Thirdly, that would disconnect me from the main community of IF authors and experimenters, and require bootstrapping an entirely new and separate community. Inform 7 is just too exciting to ignore, and it's by no means hopeless in terms of software freedom.

Summary of Software

One other thing that has caused this to get swept under the rug a little is that there are plenty of TADS and Z-machine story file interpreters out there. It's hard to explain to anyone who only ever plays IF on free software that it's all built on proprietary foundations.

The first piece would be necessary to get a DFSG-free Inform 6 toolchain, and the second would be required to make a DFSG-free Inform 7 toolchain.

Their People, Your People, and My People

As this essay is about perspectives and biases, for the most part, I am going to paint a few "us and them" caricatures to stand in for some of the positions that have been at odds here. I'll begin with my own, as that has colored all the text to this point and some readers may be baffled by my core assumptions.

My People

I consider myself a relatively dedicated supporter of free software as described and advocated by the Free Software Foundation. I believe that software holds a unique position in its status as both communication and functional work, and that this requires a rethinking of how copyright and patent law currently treat it. I am a big fan of the commons that the GNU GPL and other "copyleft" licenses have created by setting copyright on its own tail.

Since this stance covers issues of freedom of speech, education, sharing, helping your neighbor and so forth, it has become something of a moral conviction for me. Naturally, when anyone hears that you have such a conviction they will attempt to tease you with contrived dilemmas as though their very existence somehow "debunked" your moral code. [6] The interesting dilemma here is that of games and spoilers.

I have a reasonably relaxed view on games and free software, although I'm led to understand that it's one I share with Richard Stallman himself. Basically it falls under the old parenting maxim of "no secrets: only surprises". I am not saddened by an inability to easily dig out the solution to a game. The important moral question to me is the functional aspect of the game–the engine, basically. The world doesn't need to learn the location of the macguffin in your story before playing it through, but it should learn how you implemented the clever NPC engine that the story relied on.

I see a fair number of My People in the IF community, let it be said. For starters there's a fair amount of free software IF. [7] One of the big names in really clever IF, Andrew Plotkin (or "zarf", if you like) has published all the sources for his major works. Adam Thornton may be found in newsgroup archives making encouraging noises toward DFSG-free licensing and promising Debian packages when that day comes.

Although I'd note that since the Inform 6 library is linked statically into any Inform 6 work, it's unlikely that the authors who used the GPL will be able to fully satisfy a request for full source under the terms of the GPL. This sort of thing typically requires explicit exceptions noted by the copyright holder for the GPL'd portion.

Retailists

Another common perspective in these sorts of discussions is that of the software author who expects to be paid ultimately from end-user license fees. This is something of a long story, but basically in the 1980s Microsoft lobbied the US Congress to get software covered under copyright. Previously it was considered too functional to be copyrightable and too abstract to be patentable.

What resulted was a short period when people actually went to shops and put down money to take home a box of software. This still happens a lot, but it's nowhere near as total as it once was. For some people, this model is the only right and true path, and they get irritated when some punk kid who doesn't have any serious sales under his belt starts talking about sharing.

I must admit that My People have often been impolite when confronted by this perspective. It's not uncommon for a bunch of shouting to fly back and forth, and the Retailist in question fires the usual "why do you want me to starve" rant and everybody leaves unhappy.

I'm not sure how common this is in IF, but it showed up as a theme in Get Lamp a surprising number of times. Despite most of the interviews ending with admonitions that IF would never be the retail success it was in the 1980s, there still appear to be people trying to live out their dreams of becoming the next Infocom. I feel that this sentiment is at least somewhat responsible for the plague of "non-commercial use only" licenses in IF.

It is interesting to note Plotkin as a counterexample here: he recently solicited donations to sponsor him for some interesting IF projects. He asked for $8k and left the donation period open for a month, but received $15k on the first day alone. This is someone who is not wedded to the 1980s publishing model for software, to be sure!

Auteurs

Of course the Retailists and My People are kind of a sideline here. If either one had taken the fore, the licensing situation would have been extremely clear-cut. Instead what we got was something of an amalgam. The modern IF toolset was built up by a community of proud Auteurs.

I mentioned earlier that Inform attracted a lot of 1990s medialab types. These were folks who didn't come with ideas about free software, but they saw themselves as using new tools to bring forth their creative visions. They shared with one another in a spirit of free expression, but did not have the history of license development that My People had. Consider for example this outright editorial rejection of the authority of the Open Source Definition. [8]

This is where most of the disconnects seem to come from. As you dig around discussion archives and look at licenses, you find that (aside from the odd restrictions on commerce) there are two concerns on the Auteur's mind that prevent them from freely embracing DFSG-free licenses.

Being misrepresented

Artistic control

These are not unreasonable things to be nervous about, but I maintain that neither is the sort of thing that requires a hammer as large as copyright law to defend.

The issue of confusion and misrepresentation is one that many DFSG-free licenses do manage, and the Artistic License in particular requires that it be made clear when a program has been significantly modified from that provided by the original copyright holder. This is likely to be the reason why this license was chosen above all others on opensource.org.

It has been pointed out that the current license for NI and Inform 6
is arguably not broad enough for Linux norms (for instance, some
distros would not allow I6 to be shipped with their releases). We are
willing to re-address this: the decision has been postponed for
now. While Inform will probably not move to the GPL, it may well
move to an open source license granting freedom for derivative
works but requiring that such works may only call themselves
"Inform" if they comply with a reference set of behaviour -
something like the TeX license, in fact. We certainly do not
intend licensing issues to present any difficulty to Inform's
users, and the only rights we wish to retain are the moral rights of
authorship over the design of the language.

The matter of artistic control is a subtle one. It certainly ties in with the user confusion about whose software is actually running, but it goes a bit deeper than that. In one SPAG interview Graham Nelson said:

Had Inform 7 been developed in open source, I am fairly sure it
would now be an elaborated version of the superficial prototype, and
that it would be much the poorer. And it ought to be remembered
that for at least the first year of the project, I wasn't at all
sure it would ever work - "work" in the sense of being capable
enough to be useful.

I do not quite buy the argument put by Eric Raymond in "The
Cathedral and the Bazaar", that the bustling, self-organised world of
the bazaar gets things built better and faster than the secretive
clergy.

You see this conversation repeated a lot. Basically Graham is just trying to avoid the bikeshedding and keep control, and the best arguments he's getting from My People about sharing his source code under DFSG-free licenses seem to devolve into quoting Eric S. Raymond of all people.

Something that partially frustrated this was that Inform acquired a
serious user base "too soon", before the language design was mature
enough: by Inform 4, a good many people were using Inform, and this
began to make me nervous of changing the established syntax. But if a
syntax feature is clearly "wrong" (that is, anomalous and
inconvenient) then clearly one must bite the bullet and make a
change. In retrospect, I feel I have always been too hesitant over
this.

Graham Nelson knows how he works and how communities affect the way he develops software. For this reason, he is unlikely to be moved by claims of practical improvements that were originally written for a more corporate audience.

For pity's sake, dear reader, please do not take this as some sort of incitement to go and loudly argue that point. While I find the perspective behind the decision hard to internalize, it is a logically defensible position and one that stems from hackerly sentiments of history over marketing.

Pride and TeX

Right, so how did we get here again? How did we end up with pieces of this thing under a DFSG-free license, but not everything? Well, it doesn't seem that strange at all when you view the project from the perspective of the Auteur.

Graham Nelson worked very hard on a project that he feels very invested in. As an auteur, he wants to refine his creative vision for the software and then provide it in a form he can be happy with. He doesn't want his film to go out as public rushes, and he'd love it if the public's first viewing is his final edit. So he has released binaries for people to use only once the functionality was at a level of quality he was satisfied with.

As for source code, this is where the Auteur spirit truly shines. Nelson et al are using Literate Programming techniques to make a vast book out of the Inform 7 source code itself. The result will be something rather like the TeXbook in scope and presentation.

About 60% of my time spent on NI is being spent on reconstructing it
so that it can usefully be published in source, and so that others
will be able to find bugs in it, and contribute patches. I'm aware
that many people feel it should always have been out there,
whatever its condition, but I didn't think it was in anything like a
state good enough to make that productive (in terms of helping to
improve the program's internal workings), and I'm afraid that was my
priority.

Conclusion

So let's recap a little here. Graham Nelson almost single-handedly rebooted interactive fiction in the early 1990s, and a community formed around his software for IF authorship. For someone not part of the free software movement, this software was developed and released under extremely liberal terms.

As tempting as it would be to cast this community as dog-in-the-manger copyright holders dictating fields of endeavor to the world ("No commerce, kids! That's for me to build the next Infocom with!"), that characterization simply does not hold water. The Inform developers have real and practical objections to the claims of real and practical advantages to particular development styles that the Open Source movement advocates. Despite these objections they have set upon a program of releasing the entire toolchain under a free software license.

The last missing piece of this stack is blocked on a rewrite effort designed to make it fit into a self-documenting book of source code that users can really learn from by reading. If My People had this kind of dedication to the principles of communicating ideas through sharing source code, we'd be in a much better place than we are now!

I expected to research this and turn up a few stubborn holdouts who dislike DFSG-licensing for ineffable personal reasons, but instead turned up so much generosity and good faith effort that I really feel like a heel for complaining, but it's been three years.

Above I quoted Nelson in January of 2007 (nearly three years ago, as of this writing), claiming that he was spending a majority of his time polishing the Inform 7 compiler source code up for publication. The Inform 7 webs page states:

Not yet published. This is the largest and the most experimental of
Inform's components; a long-term project of rewriting and
clarifying ni for publication is now, as of April 2009, about
two-thirds done.

That update is from one and a half years ago, as of this writing. Most of the other pieces appear to have landed around mid-2008. I would really love to know how far along this project is now, near the end of 2010. Is it possible that somewhere between ⅔ and complete, there's a stage where it's good enough for public review?

I'm aware that Nelson has very strong feelings about his software that prevent him from publishing sub-standard sources. I have strong feelings about software as well, after all. My feelings, unfortunately, will continue to keep me out of a community I admire for some time.

I'm sure Graham has bigger fish to fry, of course. Inform 7 currently makes IF more accessible to newcomers, even if it alienates My People. So I sit on the sidelines, feeling somewhat helpless, flipping through my copy of Creating Interactive Fiction with Inform 7 and waiting for the day when I can join in with a clear conscience.

Hi!

Fri, 22 May 2009

Yardbird

Ladies and Gentlemen a One Mister Charles Parker, Jr.

An IRC channel I consider my "home" channel is coming up on its tenth
anniversary in a few months, and its founders have begun to reflect on
how far we've really come in a decade. By far the biggest
disappointment is that our beloved and snarky bot still runs a rickety
hacked-up version of Kevin Lenzo's 1990s classic, Infobot.

We've looked into replacements in the past, but they always seemed like
mere incremental improvements. They'd provide some degree of
reliability and a slightly saner codebase, but they generally add little
that appeals to us. Often they're written in Perl, like Infobot, which
is something we're trying to move away from (for no reason beyond the
fact that nobody in the channel is comfortable maintaining Perl code any
more).

So when I finally found a bot that managed to have reasonable gains in
terms of fun over Infobot, I leapt at the chance to put it through its
paces despite being written in Perl. For various other reasons, it was
not fit for purpose, although looking at the code led me to an
inspiration.

How Not To Write A Bot

The bot in question was written with POE, which appears to be Perl's
answer to Twisted Python. POE gives you lots of library functions and
objects that allow you to do things asynchronously by firing off events
and registering callbacks for when something finishes. This is somewhat
important, so that you don't say "Bot, do this five-minute thing" and
have it fall off the network because it hadn't returned to the protocol
code for five minutes.

But the way this bot was written was done in a very hasty JFDI sort of
scripting style. I'm told that the real marvel was how quickly it was
brought up and running, and I don't think it's fair to judge the authors
based solely on this work. However, there was something of a common
antipattern throughout the command recognition code:

Dispatchers

While tracing through a printout of this thing in an attempt to even
figure out what its features were, I thought "Gosh, wouldn't it be
great if this had a dispatch mechanism where you could associate regexes
with functions in some kind of data structure, along with some kind of
application data for context?" And of course that immediately reminded
me of...

I filed this little bit of inspiration away, and started worrying about
how I'd go about cloning this system. I'd use Twisted Python, of
course, but the asynchronous database library is on the same level as
POE's, and I prefer a good ORM. Man, wouldn't it be great if my bot
could use the Django ORM?

Asynchronicity

Of course, the Django ORM isn't built with a callback-based API, so
while your code does a query that's all your program can do. This
prompted me to wonder why that doesn't become a problem for Django Web
apps. Surely they receive hundreds or thousands of requests per
second, but concurrency never becomes a problem that exposes itself to
the app programmer.

The answer is that while Django does not have any support for
asynchronous programming, the very model in which it operates assumes
that it's being called from a Web server such as apache. Apache has its
own forking or threading model that it uses to handle lots of requests
simultaneously, and the CGI or WSGI interfaces use a well-defined
interface for passing connection information into a program and getting
a response back out.

An IRC Apache

One morning on the Underground, I began to reason that what I needed was
a sort of "IRC Bot Apache" that would handle incoming IRC events of
various sorts (PRIVMSG, ACTION, etc.), and then pass them along
with some connection information to my django code. I'd dispatch these
messages through the regex-based patterns() system, then call view code
that uses ORM objects to perform queries.

The obvious choice for implementing something like this is Twisted
Python, which shows its age but remains the de facto Python library for
coding state machines. I'm only occasionally familiar with the system
(and the documentation is filled with distracting Java-esque Software
Engineering babble for some bizarre reason), but I was able to localize
the actual Twisted-using code to one function, which at the very least
makes it simple to hand off to experts to tell me if I'm doing anything
stupid.

Yardbird

Digging through twisted documentation I found their example LogBot
and based it loosely on that pattern, subclassing irc.IRCClient and
replacing the privmsg method along the following lines:

The inlineCallbacks decorator essentially catches any yield of a
Twisted deferred object and schedules the next delve into the generator
using standard Twisted deferred-execution mechanisms. So now yield
really behaves like a scheduler yield, and you can let some more
critical IRC-parsing code run between your own calls.

Next we build a Django urlresolver object so we can dispatch regexes
to handler functions, using some Django settings info to determine
path info:

Then we build a request dictionary. In normal Django this would be
an HttpRequest object, containing all sorts of information about the web
server and the remote client and the HTTP request itself. Since this is
a quick-and-dirty example, I've reduced this to a dict for simplicity.
I also passed in the settings namespace just to be lazy (so I can keep
things like nickname in there):

Now we actually use our resolver to test the incoming message against
all our patterns in privmsg.py and return to us the appropriate
function, along with all of the anonymous and named matches that were
generated by the winning regular expression. Note that we have to
prepend a / to our message to appease the URL-centric resolver:

callback, args, kwargs = resolver.resolve('/' + request['msg'])

Finally we get to the deferred execution magic! We have a function, a
request object, and some arguments made from textual analysis of the
message. We use the threads.deferToThread method to generate a
deferred object that runs in a completely separate thread, and yield it
up to our inlineCallbacks decorator to be scheduled:

Our view function then runs in the background, taking as long as it
likes while our bot concerns itself with answering PING replies and
dispatching further events to the resolver.

We're confident that the Django code is reasonably thread-safe, as it
has to handle concurrency under a variety of Web server models (such as
apache's Worker MPM or a traditional Prefork model). Once the function
returns a value, the thread closes and execution comes back to this
method again, chucking the returned value into our response object.

We're almost done, but we still need to actually do something with this
information! In ordinary HTTP Django this would be an HttpResponse
object, containing all sorts of information on what template to render
and what dictionary to pass in as an extra context namespace. This is a
bit overkill for this example, so I've simplified it to another dict:

The various RFCs for IRC all state rather loudly that automated bots
are meant to speak using NOTICE but always ignore NOTICEs from
other sources. This is meant to prevent feedback loops flooding a
channel. Also note that since this was the final statement of my
inlineCallbacks function, I called the defer.returnValue to spit back
the result of the notice call. I'm not convinced that it was at all
necessary, but I believe it's harmless boilerplate in the worst case.

Whoa is that all?

My current version is obviously not exactly like this. I've been doing
some rather wild and thrashing testing and debugging, and there's some
mess and refactorings. The above is intended as a demonstration of the
technology only while I do my explorations in the yardbird bazaar tree.

For a start, my current version applies an errback function to log
exceptions in the view function thread, and I've refactored the above
code into a separate function that both the ACTION and PRIVMSG
functions can use. The principle is still the same, though.

What's With The Name?

Because Django is named after Django Reinhardt, and was split from a CMS
project named Ellington (as in, Duke), it's become traditional to name
Django projects after Jazz greats. For example, there's a popular
Django e-commerce system named Satchmo. I dug around and couldn't find
any named after Charlie Parker, or his nickname 'Bird'. I just decided
to play it safe and use the rarer long form "Yardbird".

Where do I get Your Version?

My spazzy tree, complete with README.txt files for the apache indexing
and a hackish approach at implementing some of the Infobot
functionality is up at http://zork.net/~nick/yardbird/. It's also a
bazaar tree, so you can just:

bzr branch http://zork.net/~nick/yardbird/

Right now I'm trying to figure out what I should really do for the
IrcRequest and IrcResponse objects, and how to properly package all this
up like a proper professional project.

Your code sucks!

Soz.

Sat, 14 Mar 2009

Your favorite ORM sucks

Over the past year or so I have been learning and enjoying Django, which is a large set of python modules that are impressively nice for creating dynamic Web sites. It's the latest generation in a set of tools that have been written to solve this very problem, going back all the way to the original CGI libraries of the mid-1990s.

You can get more details on the Django Web site, but the basic pieces you interact with are these:

Object-Relational Mapper:

This saves you from having to type in COBOL-esque SQL queries and manually connect the results to the data structures in your code.

Template Language:

This saves you from having to type in the punctuation-heavy HTML or XHTML to render your pages, and lets you focus more on the content.

URL Dispatcher:

This saves you from having to build trees of scripts or put argument-based flow control spaghetti in your code, and lets you restructure the layout of your URLs in a nice clean manner.

Form Handler:

This ties the previous three together to safely simplify one of the more dangerous aspects of programming: input validation.

There are more subsystems (such as the libraries of HTTP handling functions available to you in your “view” code), but those four seem like the legs of the table to me.

Feelings of Inadequacy

As I've been working with this, I've run into a number of conversations on IRC that seem to go something like this:

Really? I find it wonderful to have all that coordination among the pieces.

nerd:

Django's ORM sucks and is too Django-specific!SQLAlchemyforever! Behold The Power!

Okay, so what can you do? After enough of these conversations, I had this sense that I was kind of using the My First Sony of Web development systems. And the Loosely Coupled, Tightly Integrated motto has been pointed out to have the effect that while you can swap a new ORM into your Django app, few people ever use the Django ORM in anything else.

Eventually it came time to do some work on a non-Web application and I needed a database. With all these conversations with Riot Nrrds like the above, I figured I'd save myself a lot of trouble by going straight to SQLAlchemy.

The Reason We Have ORMs

There's a reason we have these “Object-Relational Mapper” things, and it's because of a problem known as the Object-Relational Impedance Mismatch. Basically, the formal mathematical model for databases used to ensure that they stay intact follows a system of tables with rows and columns and references to other tables, while data structures in most programming languages we use today manipulate data in nested tree-like structures. It's rather like the difference between a spreadsheet and an XML document, or between a ledger and a family tree diagram.

For a long time people just suffered through it: building SQL queries in strings and shoving them over the wire to databases, then manually parsing the results and populating the local data structures, all the while hoping nothing got mistranslated in the process. But in the argument over why we keep relational databases around in this object-oriented day and age anyway, people noticed that there was a formal and automatic mapping you could perform between the two.

So fundamentally the ORM is there to allow you to manipulate all your data in one single format, save you the trouble of re-inventing the necessary mapping techniques between the two, and cut down the complexity of your code thereby reducing your exposure to bugs. It's a fantastic thing, since relational databases are still important for their speed and reliability, and every reasonably-expressive programming language since LISP (which is from the 1950s, after all!) has had an implicit bias toward hierarchical representation of data.

Behold The Power!

Having fiddled with Django's ORM, and looked into Ruby's ActiveRecord system and read a bit about the history of Java's Hibernate ORM, I decided to see what was so hot about all the other Python ORMs out there.

I quickly learned that my sample of three had been somewhat biased toward a particular state of mind: namely those who want the impedance mismatch to actually be solved for them. The majority of ORMs used by the Riot Nrrd set instead seem to be written by people who enjoy the power of SQL and demand lots of advanced SQL features. Nothing to actually make your job as a programmer easier ranks anywhere on their priority list. One of the high-end python ORMS even brags in its Features list about how it can't do schema generation!

The result of all this talking out of school was that I had a project using SQLAlchemy and it was a horrible bureaucratic mess. I had session object setup and tear-down all over, manual coupling between table objects and actual useful objects for some reason, and it barely worked at all. I was forking processes off that needed to do a bit of database work, and my transactions and sessions kept stepping on each other. Hell, even the simple single-process jobs were hard to get right!

I'm sure this is where the Riot Nrrd contingent would step in and scream about how incompetent I must be. Why, a truly skilled SQL craftsman would be able to foresee all the intricate concurrency and relational integrity performance-critical session collision transaction issues that would arise, and work around them!

Back to the Tinker-Toys

Frustrated and eyeing a looming deadline warily, I decided to see if maybe I could just stick to what I knew and come up with a better solution. Hey, maybe I'd bite the bullet and set up the whole thing as a Web app inside the much-derided all-singing all-dancing framework (a term that to me typically refers to an unfinished project, rather than a useful set of libraries).

James Bennett is one of those hackers you wish there were more of on the Internet. He's knowledgeable and proficient while remaining reasonable and courteous. He reminds me of an old Jesuit brother I once knew who never seemed to open his mouth unless he was saying something that would help someone out.

Anyway, Bennett wrote an entry of his own on how to do standalone django scripts, and that was in 2007! In 2007, Django was still in version 0.96, and there was a lot of isolating of components to go before the 1.0 release. And damn if the settings.configure() trick isn't simple and straightforward!

…and then my database code lives in myapp/models.py and is something like ⅓ the length of the corresponding SQLAlchemy horror. It also gains a little input validation magic from some application-level data types (such as “IP Address”) that assert constraints not present in the database itself.

The Return of Session Management

Ah, but there comes a wrinkle! You just knew there'd be a wrinkle, didn't you? Remember how I was forking off asynchronous handler processes? So here's where that sinking feeling of inadequacy returned. I mean, Django's implicit session handling was causing me a headache, which sounds like precisely the sort of thing the Riot Nrrds were kvetching about!

Django's DB access model is very conservative: it won't chat with the database basically until the query you're building is at a stage where it needs information from the DB itself to proceed any further. It's a very lazy approach and I love it to bits. The session handling is all done behind the scenes, and any DB-bound “model” object will try to use a pre-existing connection if possible, but will open a new session if needed.

So I rolled up my sleeves and prepared to throw manual and bureaucratic session-management code all over my app, and ended up with the following:

That's it! Two lines to just drop the current database connection before forking, and one of them is an import! Then the parent and the child each get a new session the next time they perform a query or update operation on any Django model object. I'm still picking my jaw up off the floor over how simple this was.

Django For The Win

So I got to do a massive bit of coding with the delete key, and I can now be confident about the correctness of all my database code. Furthermore, I did it without dragging in any baggage from the rest of the Django set (unless you count the brief use of the django.conf.settings module, which is a fair point).

So where does all this hatred for Django come from in the Riot Nrrd set? I suspect it is partly a quick look at early (pre-0.96) versions that were still in a “we're working on separating our framework from the Ellington CMS” state, and largely a reaction against the horrible “type a bit of python code into this browser input box” model of early Zope releases. I think that Zope in the late 90s left a lot of programmers with a bad taste in their mouths.

But in the end none of the complaints about Django held up, and SQLAlchemy simply wasn't fit for purpose. I ended up feeling kind of cheated, like I'd lost a week because I listened to some uninformed Real Programmer posturing that did nothing but make trouble. It could have been that I just hit a bunch of DBAs who use Python occasionally instead of people like me who are Python programmers who occasionally need a database, but I know that's not really the case.

Okay Maybe Only 99.9%

Right now my only complaint about Django is that the #django channel on Freenode contains an op by the nickname of Magus who is the absolute opposite of James Bennett. Every time Magus answers someone's question, he does so in the most condescending way possible. Someone noticed this and set up a twitter feed to log each time he uses “obvious” or “of course”, but in February he changed his nick to avoid it. I'm sure this guy is a useful contributor to Django, but answering even naïve questions like that is simply not helpful.

Wed, 20 Dec 2006

How One-Time Passwords Work

Security experts always moan about how few people properly understand security and cryptography, and the smart ones point out that UI problems in security and crypto tools lead pretty directly to security problems. I'm convinced that the lion's share of these UI problems come from an insistance on clinging to mathematically correct metaphors that don't survive translation into everyday English.

The big example of this that I give out often is the terms used in public-key cryptography. Yes, it's true that both the public and private keys are "keys" in a mathematical sense. But one of the reasons I constantly see novices accidentally leaking private keys or otherwise confusing the two is that these names tell you nothing about their everyday purpose. In actuality, the private key is the only one that is in any sense a "key", and the public key is actually a lock.

If I want to give you an account on my system and I ask for you to use ssh keys, it's like I have a set of bicycle kennels that I am letting out for people to store their bikes in. If I decide that you should be allowed to have one, I simply ask for you to give me a lock that I can slip through the hasp on the front door and you obviously keep the key so that I never need to see it. Likewise if I give you a shell account on my system, you give me your ssh lock ("public key", to use the standard confusing terminology) which allows you to use your key (ssh private key) to enter.

That aside, one of the cleverest security access techniques to come out in recent years is OPIE, or One-time Passwords In Everything. It's basically designed so that you generate a new key each time you enter the system, like one of those constantly-changing password systems they use in spy movies. This has many of the advantages of the ssh public key system, but with the added convenience that you don't need to carry a two-kilobyte key file around with you. This convenience prevents a lot of clumsy workarounds for the key problem, which on average can be a benefit to security (if you allow long but easily remembered passphrases, for example, you avoid the problem of people keeping the short punctuation-laden passwords in their e-mail or doing something else equally dangerous).

So let's start with what the user sees when she logs into my system with the One-Time Password system. When you ssh to zork.net without an ssh key, you get a prompt such as the following:

otp-md5 73 fr8981 ext, Response:

This requires the user to use a password-generator program to generate the password. She runs the program, enters her secret passphrase and the 73 fr8981 part of the above challenge, and gets back a result like this:

KATE LAWS LESK ALOE AQUA DUMB

She then copies and pastes these six words into the ssh window, and is let in. The next time she logs in, Zork says:

otp-md5 72 fr8981 ext, Response:

and the generator gives her:

AT LYRA GEL AMOK NICK LESK

Now, it may sound inconvenient to require this password software on every system you want to connect from. If you're in an Internet cafe, for example, you don't want to have to download and install special programs. For this reason, I have set up the page at http://zork.net/ssh/ to have both a Java ssh client (on the right) and a JavaScript One-Time Password generator program (on the left). So almost any Web browser can be used to run this password generator on any machine.

This looks like it involves a lot of spooky scary hard-to-understand crypto math, but in actuality it's a super elegant and easy to understand system. So what follows is an attempt to explain the inner workings in plain English.

If you have access to a modern Unixy system (basically every major OS except
for Windows, these days. Poor bastards at Microsoft are still behind the
curve there), you'll see that there's a program called md5sum that's used to
make "checksums" of any amount of data you like.

A checksum is basically just a statistical summary of characteristics of the
data. They usually show up as a long string of hexadecimal numbers, which make
a number so large that the number of possible combinations is enough to give
one of these strings to every atom in the universe.

This is partly useful because it means that if you have a huge file and
I have what I think is the same huge file, we can each run md5sum on it
and compare the 32-character checksum string instead of comparing all
umpteen gigaboobles of data. In fact, most unix nerds use a program
called "rsync" instead of ftp or scp, which checksums pieces of each
file to send only chunks that are different. It's very fast, and if
there's an error you can just restart and it'll quickly find where it
left off. The popular BitTorrent file-sharing program makes heavy use of
checksums, and you may have seen it spit out the occasional Piece failed
checksum: Re-downloading. log message.

The really great thing about this sort of checksum, though, is that it's
impossible to go backward. You can't look at that long number up there and use
it to generate the original /etc/motd file. To do this would be like trying to
recreate the Oxford English Dictionary from a list of statistics saying how
many spaces were in it and the average length of all the words and so on. Or
it's like trying to do a sketch of someone's face using only a set of
fingerprints. It's a one-way function: lobster go in, lobster stay in.

So what does this have to do with the crazy OTP password scheme? Well you may
remember that the full string you see when you log in says something like
otp-md5 999 fr8713. That otp-md5 is there because it uses md5
checksums to verify your passphrase.

When you enter your passphrase into the password generator, it combines the
fr8713 part with it first (just to stir things up a bit, in case two people
accidentally have the same secret passphrase, and you don't want it to be
obvious by making the one-time passwords for the two users match up). Then it
runs md5sum on this combination, and gets one of those 32-character hexadecimal
checksum numbers.

Then it combines the fr8713 with the 32-character checksum it just got and runs
md5sum on that. It keeps doing this 999 times (or 998 or 432 or 73 or whatever it
said in the otp-md5 line) and then converts the hard-to-type 32-character
checksum string into those six words so that you don't have to squint so hard
to make sure you typed it in properly. The translation between the six words
and the long hex string is two-way, since the computer can easily and quickly
flip back and forth between them almost as easily as it can switch a word between
UPPERCASE and lowercase.

So remember how the md5sum process was one-way? Knowing the result of run
number 999 doesn't let you go backward to number 998 any more than those
statistics let you recreate the OED or my fingerprints give you a full-color
mug shot. And the system you're connecting to only stores the result of run
number 999 in your account settings.

So after you set your account up by entering run number 999, the first time you
log in it will say otp-md5 998 fr8713, and you'll run the JavaScript which
will do the md5sum game 998 times. You'll type the englishified result into
zork, and zork will de-englishify it and run md5sum on it one last time to
make sure that what it gets is what it has stored for run number 999.

So Zork never actually knows your secret password, but it knows how to tell
whether or not you know it. Because it knows that if it just runs this md5
checksum feedback loop thing enough times, the result will look like what it
has stored.

The OTP generator Web page I have up is all JavaScript that runs in your
browser, and the passphrase never leaves your machine. Have a look at the
http://zork.net/ssh/ page sometime, and view source. You can scroll down and
see the list of words that it uses for translating the
74d4639801fad57b90932584d1bd8646 stuff into something more typable, and then
there's a function called core_md5 that does the same thing as the md5sum
program. The rest is just loops so that it runs the checksum the number of
times you tell it, and mixes in the fr8713 and does a couple of other basic
management tasks.

Fri, 17 Nov 2006

Speak.

Sat, 07 Oct 2006

The Joy of Ikea

Very late last night I was sitting at the computer, trying to decide if anything I had to say on zork.net was worth going another half hour in the red on my sleep deficit for the week, when I hear a loud thud from the bedroom. I jump up and run to the doorway to find my husband sitting calmly in bed reading.

"Did you hear that?"

"What?" he asks, innocently.

"The thud."

"Oh, yeah. Something fell."

"From where?"

"From the bed."

"And you're not planning to get up and see what it was?"

"Must I?" He looks pretty comfortably installed, not at all inclined to get up and investigate. After all, whatever it was clearly hit the ground, so it can't fall any further, right? I, on the other hand, have decided that since he was the one sitting in the room when it happened, it is his duty to fix it.

"Well, when something falls off the bed, I think it's a good idea to at least figure it what it was. But do what you like." I go back to the computer.

A minute later I hear, "Heeeeelp!" from the bedroom, go in and find my husband hoisting the mattress on his shoulders and trying in vain to whack the side post of the headboard back into place. The bed came from Ikea Seattle, a college graduation present from my father over eight years ago. It has since moved from Seattle to Boston and Boston to Paris, taken apart on each occasion and put back together, as best as we could remember without the long-lost instruction booklet. I think its latest adventure, being dismantled completely and shoved in the bathroom while we had wood flooring installed, may have been its Waterloo.

"What do you want me to do?" I ask, quite annoyed. Now I'm sure of losing at least a half an hour of sleep with no journal entry to show for it.

"Get over there and push!" He motions to the other side of the bed. I squeeze in behind the bedstand, prop the mattress on my head, and try to get a purchase on the pine post that's slipped out of place. Two of those little filmsy Ikea-issue dowels look hopelessly misaligned with their holes, with at least two inches of space between them. I push. Nothing moves. He pushes on the other side. I, in my stocking feet, slide a foot back and hit the side of the dresser.

"I can't push!"

"You have to!"

A good twenty minutes of pushing, pulling, swearing and screaming at each other ensues, punctuated by muffled thuds as we manoeuvre the mattress around the room. At one point I'm completely blocked into a corner with the mattress leaning against my back, and I start laughing uncontrollably. Any remaining force I have is drained from me instantly. Comedy becomes tragedy a few minutes later when we knock over a lamp and break the lightbulb.

My husband finally has the bright idea of removing the bed slats before trying to pushing the headboard back together. Genius! At one o'clock in the morning GMT + 2, the whole thing is miraculously back in one piece. We've even profited from the lightbulb incident by finally getting all the dust bunnies (moutons, or sheep, in French) vaccuumed up.

Of course, I have absolutely no confidence that the bed will hold together for more than a couple of months. Ikea furniture has the bad habit of falling apart at the most inopportune moments, as on the morning my husband sat down to breakfast and had the kitchen chair collapse underneath him.

Meanwhile, I can only imagine what all this late night activity sounded like to our neighbors. Though the average Parisian has likely had similar Ikea adventures, I couldn't help but blush thinking about it.

Thu, 05 Oct 2006

Prise de Tete

Prise de tete literally means "taking one's head," in the same way one could take a city or a castle. That's the accurate-yet-colorful translation: in everyday English, it means something like boring, frustrating, a waste of time or a pain in the ass. And as one would expect of such a useful, all-purpose term, it has got to be one of the most common phrases in modern French.

Examples:

"Je voulais aller au cinema, mais c'etait trop prise de tete." I wanted to go to the movies, but it was too much of a pain in the ass.

One prime example of "prise de tete" is the French Office Greeting Protocol. It is very important to properly say hello to everyone upon your arrival in the morning. A simple "hello everybody" from across the room or a nod and smile while crossing in the hallway simply won't do. You must go around from desk to desk, shake everyone's hand and say hello to them directly and personally.

You must know in advance if the person in question gets a "bonjour" or the more familiar "ca va." This is a question of hierarchy and familiarity; I can "ca va" my boss, but not his boss. You must also be prepared to say hello to the person with their first name, as in "Bonjour, Philippe," but only if they do so first, and only if you're damn sure you've gotten the name right.

Half of the project managers in my office are named Philippe, so it's a safe fallback name when in doubt.

I tend to freeze and splutter something incomprehensible when someone addresses me out of the blue. I'll suddenly lose their first name, even though I've worked with them daily for over two years. Or I'll get so tripped up on the "bonjour or ca va" question that the name will come out after a long enough pause to make my colleague wonder if I'm suffering from early memory loss.

It isn't something that probably matters much really, but it's yet another example of an everyday script in my life here that I'm always feeling I get just a little bit wrong. On the other hand, I felt that way often enough even back in the US, I'm relieved to at last have the perfect excuse.

Don't mind me, I'm not from around here.

Speaking of prise de tete, I've tried to get the unicode character references to work without success, and I'm reluctant to mark all my text as raw html just to add in a few accented characters. Anyone with more experience or more hours of recent sleep is welcome to help me out.

Fluctuat nec mergitur

The motto of the city of Paris is a, to me, nearly unpronouncable Latin phrase that means something like "it is buffeted by the waves without being submerged." That is my literal, half-past-midnight translation from the French I found on the French wikipedia site. I prefer the way my husband translates it, "Et portant, elle flotte." And despite it all, she floats.

Paris is a city, and thus a feminine entity in French. I think that's why I prefer this last translation, which keeps the feminine pronoun, since I often feel I could adopt the motto as my own. Wikipedia also informs me that this motto comes from the Scilicet, the ship represented on the city's seal and symbol of a merchant's guild in the middle ages. The ship is almost unrecognizable as such on the city's modern logo, but it nevertheless has at least a half dozen centuries of history.

As of August 24th, 2006 I have three years of history as a resident of Paris. I don't even live in the city itself so I'm somewhat of a faux parisian, but more on that later. It's been almost eight years since I first visited here, in late 1998. Perhaps I've not much to tell yet.

Thu, 18 May 2006

Shipping

Of course, shipping couldn't be easy, now could it. The calculations
for the USPS seem pretty simple: you just punch in a weight, and they
have some formula for calculating the maximum allowable volume. Hell,
it may even be constant.

But UPS has formulae that make the tax schedules look elegant.
Basically, they calculate a weight that the box is supposed to be,
based on volume, and the weight that the box actually is, and choose
the higher one. So you get hammered hard on packing materials and
double-boxing.

And you know how UPS domestic has like ground shipping where you just
pay a minimal amount and let it take forever? Well overseas it's
entirely impossible to distinguish one class of service from another.
Near as I can tell it's just like random priority froofraw.

And then there's the old "insurance vs. customs" game. If you insure
your package for lots of money, customs may make you pay that in full
just to let it enter the country, effectively costing you the entire
value you insured against loss. Lose-lose!

So I kind of scratch my head and go "um, none of the above?" and they
drop me like a hot rock.

I spoke to a longtime friend who has done the expat thing many times in
his life. According to him, oil company stooges are the worst. You'll
walk into a merkin's house in Lagos and there will be all the family
antiques. He goes to parties in penthouses where dining tables have
been hauled from Nebraska and fill all available space in whatever room
they're stored. Basically some midwestern housewife is handed an
expense form by her husband, told "the movers will be here on Tuesday,
so just show them all our stuff" and then sort of by default has them
ship anything not nailed down. The moving companies count on this.

These people run 110V heavy appliances like washing machines and
refrigerators from foreign power, just using a converter. What, they
don't have refrigerators where you're going? Is your dishwasher
really that precious to you?

Add to that the fact that moving and shipping companies seem to all be
crooked (been looking at http://movingscam.com) and the whole thing just
seems so not worth it. At this point I'm ready to just pack it all up
myself and ship UPS and USPS.

What a goddamn racket!

Tue, 09 May 2006

Success.

Fri, 28 Apr 2006

Choose other peoples' words wisely

So, on the outside walls of the cubes at work we have these little whiteboards for letting people know that we're on vacation or whatever. Most of the time they go unused, so some people have gone to putting inspirational quotes on theirs.

Tue, 11 Apr 2006

Filling The Sink

When I first learned to shave, I was still a teenager living with my
parents. I learned to shave using the home sink with its seemingly
limitless supply of hot water. And my father taught me to just rinse
off the razor in the stream of water as needed, to flush out the
harvested whiskers.

But based on all the forums and shavegeek photosets and all, it seems
that the preferred method is to fill the sink with hot water, and just
go from there. The advantage, they say, is that the silence of the
still waters will allow you to hear the rasping sound as you work. This
promotes keener understanding of the whole process, and results in a
finer shave.

The only problem with this is that I used to fill the sink to shave,
years ago. I had no choice, until we called the plumber to clear out
the pipes. It was really distressing, since I'd leave a ring of hair
and soap around the bowl, and I just found the whole process gross. I
came to associate "full sink" with "filthy sink".

But since our drain has worked nicely for the past couple years, I
decided to give it a try a while back. After all, our dinky
energy-saving apartment water heater only has a couple minutes of hot
water left once I'm done with my shower.

Boy did I have flashbacks. I spent a good five minutes working to clean
the Taylor's Lavender Cream residue off the sink rim, wiping away
whiskers with my finger. Hulk SMASH.

But this morning I decided to give it a go again, this time using the
Proraso pre-shave cream instead of the nancy boy pre-shave oil. I
lathered up with the Proraso soap on top, and got to work.

Imagine my surprise when all the little globs of hair and cream stayed
coherent in the bowl as I shook them off. It didn't last, but the hairs
didn't all migrate to the meniscus or cling fast to the sides of the
sink. When I was done, all it took was a little splashing with cold
water and the whole lot went down the drain, quick as you please.

And despite the double-numbing mentholyptic treatment of both proraso
pre-shave AND lather didn't seem to matter. I got a great shave without
marks or irritation. I could hear what I was doing, and began judging
my shave based on the sounds it made.

Hey Nick, if you want a tour round Bath sometime let me know

Troncon On The Rock

It's Happening

And there ain't a gott-damb thing you can do about it!

Don't believe the hype spewed by the Liberal Media! They may sound
convincing with their snooty tracts named such as Treatise of
Government and Common Sense, but do not believe their
Constitutionalist lies! These nefarious trolls have missed one key
and essential fact that changes the very equations of our lives, our
fortunes, and our sacred honor! The Monarchy will never die!

That's right, folks, I am indeed moving to London. And that means it's
time for one last Troncon in San Francisco before I go.

May 2006

Su

Mo

Tu

We

Th

Fr

Sa

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

The Dates

The shindig is set for the first weekend in May. That's May 5, 6, and
7. This has been set in the #tron topic for the past week or two.

This date was chosen because the largest number of #tron-ites were
available to come.

The Attractions

One of the reasons that this calling-of-the-tribes occurred is because I
will no longer volunteer for the National Park Service past May. Come
June, I'll be all wrapped up in the move to London, and won't be able
to spend any time on the island. So this is your last chance to get
the super secret behind-the-scenes tour of Alcatraz!

As a volunteer with the National Parks Service, I have the keys to take
folks into areas that the general public don't have access to. This
means that instead of the quick-and-dirty half-hour audio tour of the
cellhouse that most of the "it's Thursday so this must be San Francisco"
crowd get, you'll receive the full red-carpet historical interpretation
of a trained NPS docent.

Allowing for various time constraints, I can take folks into an old US
Civil War-era tunnel, the disused A-block area, the hospital, and
possibly even the old gun gallery. Bring your cameras, because the
photos practically take themselves.

Tickets

Since I'm doing this as an NPS volunteer, I have to put in a full day's
time even if folks show up for a half-day. However, I recognize that
some people will not be in town until Saturday morning and can't make
the 9:30 ferry. So what we're going to do is a two-staged event.

All Tronconners will need to purchase their own tickets for Saturday,
the 6th of May, 2006. The full-day crowd will buy tickets for the
10:45 ferry, which will likely be met by me on the island dock. The
rest of you will catch the 12:15 ferry at the latest, and will join
the party already in progress.

Alas, the Blue and Gold fleet's on-line ticket system only sells
tickets with the audio tour price included. If you want to purchase
a slightly cheaper ferry ticket that does not include the audio tour
rental fee, you will need to call their ticket office on the telephone.
The number for that is (415)705-5555.

If you are going to be on the 12:15 boat, it is your responsibility to
find contact information for someone from the 10:45 group. There are
payphones outside the dock restroom building, and the island has its own
mobile telephone repeater. Barring that, the ranger or volunteer
staffing the dock will have a radio that can reach me. Just tell them
that you're trying to meet up with "an NPS Volunteer named Nick". If
you find yourself woefully tardy, they'll probably send you up to the
top of the hill to meet us outside the cellhouse.

Once again: Purchase a 10:45 or 12:15 ferry ticket for May 6th,
as fits your schedule.

Once you have purchased your ticket, let me know which ferry you will be on.

Rules

The site's official rules are simple:

Eating and drinking are allowed only on the dock area at the bottom
of the hill. The only thing you will be allowed to consume past the
dock will be bottled water.

You will be required to help preserve the site by not taking any
idiot souvenirs or leaving marks on the site. For criminy's sake, if
you have to have a chunk of the island, they sell Pet Rocks in the
gift shop.

Don't mess with the seagulls. It's their island. We just visit it.

Unless I'm explicitly lowering a divider and ushering you through,
don't cross any signs that say things like "restricted area" or
"closed for your safety".

Now, unofficially, I have a few reminders for you. The entire island of
Alcatraz is a National Park. That means it's owned by the National Park
Service, which is under the Department of the Interior. It is Federal
land. The Law Enforcement Rangers and Park Police may look like
rent-a-cops in RCMP hats to you, but once they catch you in the wrong
you'll be stuck in the federal court system (and probably so will I).

So please, leave all the counterculture contraband in San Francisco or
Oakland, where the local constabulary have a history of being more
accomodating to that sort of thing. If I were you, I'd even leave the
booze at home.

Lunch

This is an important point. Since food and drink are only allowed on
the dock, you'll want to consider two options for your lunch situation:

If you are on the 10:45 boat, you may wish to pack a lunch that you
can carry around the island. We'll head back to the dock for a
furtive picnic and to wait for the 12:15 folks, dodging seagulls and
pigeons all the while.

If you're on the 12:15 boat, you will probably want to eat before you
board or on the ride over.

If you're looking for places to pick up food quickly on the way to the
island, I recommend the Boudin bakery by Pier 43 (across a parking lot
or two from Pier 41). But they do sell factory lunchoids on the ferry
(pizzas, stadium nachos with pasticine cheese-product, pretzels, etc) if
you're tight for time.

Transit

Pier 41 shares a Muni stop with the tourist trap Pier 39. The most
pleasant way to get there is to take the F line streetcar. The F line
runs down Market Street, and thus intersects BART and Muni Metro
stations neatly. Plan on a charming ride of at least half an hour, and
also plan to show up 15 minutes before your scheduled ferry departure so
that you can find the place and line up.

If you miss a ferry, the standby line is an unpleasant experience, and
you'll need to call someone in the group to let us know you'll be late
by a ferry or two.

From the Airport

If you're coming in on Saturday morning, bear in mind that San Francisco
International Airport isn't even in San Francisco County. It's out in
Millbrae, and while BART does connect via the International terminal, it
will take a while to reach the city proper.

Other Preparation

My last bit of advice to you is to ignore anything you think you know
about weather in California. San Francisco in the late Spring/early
Summer is a schizophrenic cycle of calm sunshine and windy fog. And the
weather on Alcatraz is the weather of San Francisco, amplified.

So the wind and fog that glances The City will blast right across The
Rock, and the sunshine reflected off the bay has left me with more than
one sunburn on the underside of my nose. Dress in layers, so that you
can cover up against wind and rain or strip down to your shirtsleeves as
the day requires.

Lodging

I've never had to find a hotel or other short-term lodgings in San
Francisco, so I'm afraid I can't be of too much help. Some #tronners
may be willing to host guests, but most of those available are likely
already booked.

One person, however, has come upon the novel idea of staying at the Fort
Mason hostel. I can't speak to the quality of the stay there, but it
has the advantage of being just on the other side of Ghirardelli Square
from the high-numbered piers. It's an easy downhill walk from Fort
Mason to Pier 41 (but leave early), and you have the advantage that it's
the site of the local NPS offices. Heck, it's part of the Golden Gate
National Recreation Area, just like Alcatraz!

Tue, 28 Mar 2006

The Earthquake Centennial

I have of course said this elsewhere, but it bears repeating.

On the 18th of April, 1906, at 5:12AM local time, a colossal earthquake
hit California, centered on San Francisco. This is one of those events
that defines SF history, and so every year there's a celebration in the
early morning.

Of course, since this is 2006, this year's will be enormous. People are
making a big to-do about how the hundred-year anniversary is coming.
But there's just one problem:

The quake happened in 1906, before Daylight Saving Time. The quake
happened at 5:12AM Pacific Standard Time. The revelers this year are
celebrating at 5:12AM Pacific Daylight Time. They're an hour early.

If that doesn't convince you, consider that the accounts of the quake
always describe the "early dawn light" that illuminated the city when
the quake hit. A sunrise calendar I just used lists morning twilight as
6:03AM PDT, and sunrise proper as 6:31AM PDT. The sky is still black at
5:12AM PDT!

Naturally, as with the whole "2000 is still the 20th Century" crowd, all
this amounts to is proof that I'm a smartass.

Sun, 26 Mar 2006

Proraso Pre-Shave Cream

Okay, so I hit the shavegeek forums every couple of months just to see
if they've come up with anything interesting beyond "OMFG TEH BURBLE
SOAP IS LESS SANDALWOODY THAN THE GURBLE SOAP I H8 U". And a couple weeks
ago they were all abuzz about the famed Proraso brand Italian shave soap being
carried in Target of all places.

I used to troll Italian import places in North Beach who laughed at me
to my face for asking about this stuff: "Oh, you'll never find that
around here!" It's an anomaly in shavegeek accoutrements. All the
ingredients say wrong wrong wrong: it's mentholated, confabulated, and
extracted and yet it is somehow a formula that wins real fans. If you
thought the Nancy Boy altoid effect was strong, well Proraso's menthol
and eucalyptus is like having the Halls of mentho-lyptus all over your
face.

So I wrote to Pedro because there was a big fight on the forums about
the US formula. Supposedly the ingredients list for the Target stuff
doesn't match the imported stuff. Lots of European shaving cream
manufacturers have been reformulating to meet with European Union
FDA-type regulations, but we don't really know why they chose to do
this. However, the rundown of all those threads is that the one
change they made was:

...to ditch Lanolin and replace it with sodium borate.

This was good news, since Uncle Pedro is allergic to lanolin. It
probably also makes Proraso a vegan formula, which could be handy for
lots of other folks.

So I shaved with Proraso shaving soap a few times, and it really felt as
though my face were going tingly-numb. It has a strong medicated feel
to it, and I think that I got a little careless as a result. The shave
I ended up with was rough and overscraped, and I really couldn't tell
how brutally I was working myself over with this anæsthetic stuff on.

Now, it's probably fantastic stuff for the summer. The cooling effect
would be such a great thing on a hot day that I'd probably dab the stuff
behind my ears and along my neck for relief from the heat. But I think
I'll leave my little $4 tub of the shaving soap alone for now.

However, lots of shavegeeks rambled and raved about the pre-shave cream.
Some folks groused that it was no better than a "proper prep" (which I
assume means seven minutes of hot towels and a deep-tissue massage), but
others swore by it. So I figured what the heck and went back to Target
to get a jar.

So this morning I put the stuff on under a layer of Taylor's avocado,
and did a quick shave with a four-day-old blade that had already been
giving me grief. It's possible that I just mixed the lather a little
richer than usual, but the shave was flawless. The metholated cooling
was more muted than when I used the proraso soap, but I still felt like
my face was a little tingly-numb. Still, I emerged baby's arse smooth
without so much as a single bump.

I'll definitely try a bottle of Nancy Boy pre-shave oil to compare, but
I think this category of goo may be the next big thing for me in blowing
petty cash on vanity grooming products.

Shaving Ads 1914-1953

They cover both world wars, and it seems interesting that in the earlier
images they seem to really be hawking shaving sticks. In one of the
earliest entries, however, The Conversion of Mister Prejudice, they
show someone applying it to his face and lathering against his chin with
his brush. I've never used a stick, but I always thought it was
supposed to be brushless.

Lots of brushless creams show up in the post-WWII entries, and powders
seemed big in the 20s. I have no idea how good any of these shaving
products were, but the brands represented really are terrible nowadays.
Barbasol is really the industrial foam goo of last resort.

As always, implications of getting laid by being smooth-shaven abound.

And of course, you can't have a collection of old shaving ads without a
few Burma Shave rhymes.

Wed, 22 Mar 2006

The Lock Master

Gary,
I read the lock hacking bit, and I was surprised at how simple the
method was. It's so obvious once you know it, I'm a bit mad at myself
that I never realized this years ago. Converting an m^n problem to
an m*n problem simplifies things wonderfully.

Of course, once you come to the hack, the next question is how to
prevent that exploit. The main way I can think of is to find a way to
tie the pins together so that they will only open when all of the pins
are set for master or all set for the lock-specific key. This
solution would probably lower the number of possible keys, as well as
making the locks more specialized. (One advantage of the susceptable
master keys systems is that they can be made from regular single-key
systems).

Of course, I'm having trouble finding key blanks which match my dorm
keys online, and I don't want to walk into a hardware store comparing
random key blanks with a key stamped:

U of M
DUPLICATION
PROHIBITED

Tue, 21 Mar 2006

reST and HTML woes

Well, I went through and cleaned up my old entries, using this
technique. When I was done, I decided to clean up all the entries
that were generating reST errors because reST was freaking out trying to
interpret raw HTML. I basically went through and cleaned up all of
Nutella's entries and a bunch of Sam's.

For Nutella, he essentially wrote everything as one solid block of HTML,
often with no newlines at all. So what I did was wrap most of his
entries in reST's raw html directive, like so:

Nutella used <cite> a lot, and decimal unicode entities to print hanzi.

For Sam, he had written most of his entries in the wiki markup, using
only occasional HTML for URLs and <pre> blocks and so on. I wrapped a
few individual paragraphs in the .. raw:: html bit, but mostly I
just fixed his bad entries to be properly reSTy.

A Few Good Tags

The switch to reST may be preventing some folks from using this, so
here's a few basic tips to cut most of the noise from the quickref.

First, pre blocks are handled using a double-colon and some strict
indenting:

::
Note that
there is a blank line
between this paragraph
and the ::.
It won't stop being <pre> until we outdent back flush
with the ::!

...becomes...

Note that
there is a blank line
between this paragraph
and the ::.
It won't stop being <pre> until we outdent back flush
with the ::!

Next up is hyperlinks. The finnicky bit about these is that they tend
to behave more like footnotes or endnotes. Basically if you end a word
(or a string enclosed in backticks) with an underscore, it becomes a
link to a URL specified later on (after the paragraph, or at the end of
the file, or wherever you like):

The switch to reST may be preventing some folks from using this, so
here's a few basic tips to cut most of the noise from the quickref_.
.. _quickref: http://docutils.sourceforge.net/docs/user/rst/quickref.html
First, `pre blocks`_ are handled using a double-colon and some strict
indenting::
[...]
.. _pre blocks: http://docutils.sourceforge.net/docs/user/rst/quickref.html#literal-blocks

Then there's images. This is handled with the .. image:: directive,
as follows:

Finally there's Unicode. There are a few ways to do this. For example,
in Ubuntu I use SCIM to input UTF8 text in any application, so I can
just hit ctrl-space and start banging in 日本語. But some folks don't
have that luxury, so here's a few ways to do this:

日本語 is 日 (sun), 本 (origin), and 語 (language).
Thus, 日 本 語 is "the language of the rising sun".

But again, you can always just go back to HTML character entities and
wrap a paragraph in .. raw:: html.

The rest is uncommon enough that you can just consult the quickref or
look at the .txt file for an entry you see doing what you want. Also,
feel free to pop into nwall and ask for help if you're trying to do
something advanced with reST.

But most of what you do will just be ordinary plain text paragraphs,
separated by blank lines.

Come on Team

How to Fix Old Entries

So one somewhat bothersome aspect of the blosxom system is that if you
edit a file, that bumps it up to the top of the listing. This is
because pyblosxom sorts all entries by the modification time of the
original posting.

There are quite a few old entries in here that could do with a few
repairs, but they really don't belong at the top of the page or the RSS
feed. So for you folks, I give you the magic of touch!

Here is an example of how to use cp and touch to edit old
entries while keeping their posting date the same:

First, the cp -a causes the file in /tmp/ to have the same exact
modification time as the oirginal file. We are actually just backing up
the modification time here, and don't care about the contents.

Then we edit the file as normal (I use vi, but I know that many folks
like nano).

Finally, we use touch -r to make the edited entry have the same
modification time as our backup in /tmp/.

So this would be helpful for folks like Nutella, who need to go in and
mass-replace a lot of <br> and <p> stuff with ordinary blank lines.

If you edit an old entry, let me know and I'll give the full site build a push.
Right now pyblosxom only updates files that have brand new mtimes, so this
trick will sneak past that. I can force a full rebuild that will update
everything.

Sun, 19 Mar 2006

Fuck Everything, We're Doing Five Blades!

The Economist drops a little science on the diminishing returns of
multi-blade cartridges:

It is simply not possible to add a new blade whenever the
marketing department wants one. Every additional blade, explains
Michele Szynal, a spokeswoman at Gillette, adds weight and size
to a razor. Firms must therefore find ways of making both razor
and blades lighter, which means thinner blades, more closely
spaced, made of special materials, with new coatings.

Thu, 16 Mar 2006

More Pyblosxom Magic

So we now have the latest 1.3.2 Pyblosxom, which means we have proper RSS now.
I've updated the little XML box, but the long and short of it is that you
should change "index.rss" to "index.rss20" for your feeds.

a strong technical argument, economics, a certain historical
chic, and the aura of self-confident masculinity without all
that messy bravado

For the next week, I did all the usual Internet research you'd expect before
going off into some goofy "lifestyle" craze. I read Corey Greenberg's
The Perfect Shave (which is still required reading, even though Corey
has moved on a little since then), trolled through the Shavegeek
Forums, and spent hours comparing products on Classic Shaving. I
made some rather nice purchases based on that.

However, I've learned a little since then, and that's the main reason
why I'm posting this here: so far Sneakums, Octal, and Uncle Pedro
have all joined in the fun with old-timey shaving gear, and more seem to
be following. Last Autumn, I'd have just thrown those three links at
them and let them read up on it like I did, but now I'd like to explain
what I'd do if I were getting into this today.

The Gear

So first of all, just to avoid any confusion on this matter, this is
not about straight razors. The trade nickname for those is
"cutthroats", and I'm with Corey Greenberg when I say that they're just a
step too far for me. What I'm babbling about is what people used to
think of when you said "safety razor" before disposables and
expensive cartridge-head monstrosities reached the ridiculous point in
history when they said "Fuck everything, we're doing five blades!"
(yeah, that used to be a joke. I know.)

So instead of a wicked blade like this one:

I am talking about something more civilized like this:

The razor is a dual-edged safety razor, and you load it by unscrewing
the bottom, lifting up the curved hammerhead portion, and sliding a
dual-edged version of a surgical razor blade onto the spindles inside.
Sweeny Todd, go home!

Why isn't it better to have five vibrating blades in a single head like
Gilette says? Well, there are a number of reasons most people cite,
such as the cheap factory production of the multi-blade cartridges, or
the tiny gaps between blades clogging with hair and dead skin. But in
general the multi-blade razors are trying to emulate someone doing
multiple strokes with a single-bladed razor. The results are mixed.

The one thing that takes people by surprise is how short the handle is.
It's about half the length of a disposable razor's handle, and you hold
it with only your fingertips. I find that it gives me much better
control, although you can buy long-handled DEs.

The particularly astute among you will notice that my razor is sitting
next to a rather large brush that boasts the absurd-sounding category of
Super Badger. Wind in the Willows it's not.

The brush is due to the fact that I ditched cheap aerosol cans full of
shaving gel or foam in favor of luxury imported creams. And you know
what? The import creams are cheaper and last longer and are so much
more enjoyable than the pressurized gel I used to use. They're two
parts shaving lubricant and one part skin care product.

Take a look at the jar on the left:

That's the Taylor of Old Bond Street Avocado shaving cream. It's a
widely-respected favorite due to its heavy use of avocado oil, which
lubricates and moisturizes.

Shaving

Many of the articles on "wetshaving", as the fad calls itself (what,
were people really shaving dry before?) act as though the reader is a
complete n00b who never learned how to shave at all. To hear Corey
Greenberg and the Shavegeeks tell it, millions of men are grabbing cheap
disposable razors and just mowing into their cheeks without even
bothering to head to a sink or anything.

When I saw Predator for the first time, and Bill Duke pushed that
blade until he bled, it was the only part of the movie that scared me.
But I guess I must be in the minority, because I actually had my dad
stand me by the sink and teach me to shave using a dual-bladed razor
sample that was automatically sent to me around my 16th birthday
(suspicious, but my mother never cared much for privacy, and signed all
of us up for who knows how many sucker mailing lists).

So I learned to wet my face and work with the grain before going against
it. I also had my first shave in the chair of our family barber, his
expert hand scraping the weeds and peach fuzz off with a deadly
cutthroat. All the same, I retreated out of fear to an electric device
(derisively referred to by the fanboy shavegeeks as a "lawnmower") for
many years. The acne only cleared up when I finally went back to a
blade.

So there's a simple set of steps I go through to shave now, which is
really all there is to the goofy term "wetshaving". Corey Greenberg
acts as though it's a radical departure from the norm, although I think
it's basically the same way I shaved with a Mach 3 and gel.

Wet face with warm/hot water. This opens pores and softens hairs. A
hot towel is ideal here, but you can just take a hot shower first and
not dry your face.

Build and apply lather. This is done by soaking the brush in hot
water and letting it drain, then swabbing the tip with cream and
whipping it in a circle in a mug or directly on my cheek.

Shave along the grain, applying very little pressure.

Repeat steps 1-3 if needed, going against the grain if necessary.

Cool down by rinsing with cold water to close the pores, and maybe
add an after-shave toner (that's something gentle like witch hazel,
not some stinging rubbing alcohol perfume nonsense like Old Spice).

Now the shavegeeks shout loud and clear OMFG DO NOT APPLY ANY PRESSURE
WHEN SHAVING. Of course I apply pressure when I'm shaving, but I
sure don't shovel away like Duke up there. The way folks like Corey
Greenberg make it sound, everyone's just spraying gel onto a dry chin
and then pushing against their face like they're pressing air bubbles
out of wallpaper.

The one thing I will say is that if you do go against the grain,
that's when you have to barely tickle the surface with a feather-touch.

My Kit

So you've seen the photos, and this is what I have, for better or worse.
My first order was from Classic Shaving, and consisted of:

I've been fortunate in that there's even a cigar shop on Market that
carries Merkur shaving supplies without egregious markup. Nancy Boy is
a local company, and folks pay good money to have their stuff shipped
all over the continent (and their laundry detergent is the best I've
ever used!). Most Target stores now carry Proraso in their "Spa"
section, and I grabbed Lucky Tiger at Elephant Pharmacy and I've seen it
at Whole Foods.

What I'd Change

Now, let's look over my original Classic Shaving order. The Merkur
HD (or "hefty") is a classic, and one that shavegeeks seem to return to
time and again. I like that the extra weight gives it some mass, so
shaky fingers aren't a problem. When Uncle Pedro ordered, they were
out of all the hefty Merkurs except the open-comb version, so that's
what he uses. The open comb seems to be good for thicker hair, although
it tends to shave a little closer, I'm told. That might make it not
quite ideal as a "my first DE razor", but it's still a fine instrument.

The Taylor's Avocado is good stuff, although I think I'd go with rose
rather than lavender for my second tub next time. The blades are
something I haven't experimented with, and I'm still using the Merkur
Platinums.

But the brush is something where I feel like I'd make a different choice
today. The shavegeeks are all about huge enormous brushes that carry
six faceloads of lather in a single dab, but they've taken it to excess.
The 2235 is the model that appears in The Perfect Shave, but Corey
Greenberg himself recently said that the 2234 is his favorite overall
brush: it's $10 cheaper, and a little bit smaller and easier to manage.

I do feel a little out of control with my big sloppy brush from time to
time, but Corey has gone on to the Wee Scot mini-brush now. He's
basically using little travel brushes to lather these days, so who knows
what that's all about.

No Regrets

Of course, I'm not about to buy another brush. Buying a brush made of
badger hair kind of went against my vegetarian sensibilities as it is,
so I'm not going to waste what I have. Also, the brush was half the
cost of my initial purchase to begin with. My reasoning was that the
razor and the blades were such a small portion of the order that if I
decided that shaving with a DE was too rich for my blood, I'd still have
a classy brush and creams for use with a pedestrian Mach-3.

I will say that the larger brush does hold a good deal of warm water,
and it's great to get that first pass of warm lather on the cheek.

What I've Added

The Nancy Boy is another big favorite, and I loved being able to just
walk into the Hayes St. store and pick up a jar (oh yes, and do try
their laundry detergent!). They're a super friendly company that
seems to know how to win repeat business. I'm not sure the shaving
cream is as good as the Taylor's Avocado, but it's not really in the
same category.

Uncle Pedro described the NB cream as "like an altoid for my face"
due to the mix of lavender, peppermint, and rosemary oils in the
formulation. His only comparison was Taylor's lemon/lime, which left
him nonplussed.

But speaking of altoid-on-the-face, I finally managed to grab a tub of
Proraso shaving soap. It's another one of the brands mentioned in The
Perfect Shave, and it's got menthol and eucalyptus to make a crazy
chilly numbing tingling sensation like medicated shaving cream. I'm not
sure it's the best thing for a chilly winter morning, but I mean to give
it a chance. It's probably wonderful on a hot day.

But the best part is that with a good brush, just about any decent hard
soap can make shaving lather in a pinch! In fact, a good chunk of olive
oil soap is one of the more popular hard shaving soaps, and you can use
it to wash your hands or feet or use it in the shower or whatever you
want when you're not shaving with it. This gives me a sort of rugged
self-reliant confidence, like I could McGyver up a shaving setup in the
field if I had my brush on me.

Finally, I bought Lucky Tiger aftershave strictly because Tom Waits sang
about it. I know he was singing about hair products made by the old
company, and I bought 1990s-inspired nuts-and-berries New Organics
stuff, but that suits me fine. I get pure aloe and orange extracts and
chamomile and witch hazel and all that good stuff, and in a classy
looking retro bottle made to look like the old brown glass pharmacy
vials. It works a champ, too.

What I Recommend To You

Okay, so after all this, you're squirming in your seat, adjusting and
readjusting your ironic horn-rimmed glasses, crying out "Oh, but now I
simply must get in on this hip new retro craze! Tell us what to buy!"
Your consumer obedience circuits are shorting out! Just sit back, take
a stress pill, and think things over.

If I were to place an order from Classic Shaving for someone new to
this, here's what I'd get:

Uncle Pedro ordered a set like this, only with an open-comb HD and
Lemon-Lime Taylor's instead of avocado. I basically told him that the
citrus creams tend to be made for oily skin, and the avocado is best for
dry. Well ol' Pedro slapped the dust from his rough-cut hands, and
gesturing with his John Henry mallet he proudly informed me that he had
"combination skin". And we left it that way as men, true to our word.

So I sent him a travel jar of Nancy Boy Shaving Cream, prompting his
gushing "altoid" comment. The Nancy Boy is made with Avocado oil,
although it's not the all-hallowed balm that Taylor's is. Still, it's a
fantastic all-around shaving cream, and you could do so much worse and
still be in the top shelf.

Aha, you say, but you are in San Francisco! You demand instant
gratification! You don't want to wait around for some Angeleno
importer to ship you your gear via UPS GroundSloth! You want to pound
the pavement and return home with bags full of gear!

I am powerless to resist your consumer gusto! Demand no further!

I have no idea how you intend to comparison shop on brushes, so you'll
just have to comparison shop a bit in person. But the rest of the gear
is available within easy walk of BART stations.

Grant's Tobacconists carries merkur razors, unlabeled and
uncategorized. The shopkeeper is an old Boer who knows two things
about the stock: Diddly and Squat, and Bo Dilddley's tour left San
Francisco months ago. I occasionally go in to buy another box of
blades (mostly because I can, since it took me six months to make it
through the first box) I ask to see the Merkur Platinum Blades and
he always responds "Uh, I think they're all made of steel." They've
got mugs and brushes and other things, but I think they only have
boarbristle instead of badger.

They're by the Montgomery Street Station, North exit, across from
Stacey's independent bookstore and next door to Patrick & Company
stationers.

Nancy Boy moved out of their Castro digs and into a cute little
shop in Hayes Valley. Stop by and pick up a jar of the Nancy Boy
Shaving Cream and a tub of the laundry detergent. If you sign up
for their mailing list, they give you a discount right then and there.

But generally stay away from the haberdashers and "gentlemen's
clothiers" you'll find downtown. Most of those guys sell the imported
English creams like Taylor's, but at stratospheric markup.

Mon, 19 Dec 2005

The Wonders of Pygame

After spending a good amount of time in the SAIS source code and getting a
feel for the game logic, I sent a long mail to a collaborator with my
dream goals for the system. I listed a bunch of things, but my real
pie-in-the-sky items were these:

Make solar-system navigation in something akin to the tactical mode.

Make in-game object rules handled via embedded scripts using something
like lua or embedded python. This would make it a modder's paradise,
with new objects and graphic sets easily spliced in.

Add planets and station objects in the solar-system view.

After poking around through lua some, I chanced to take another stroll
through the pygame docs. I had not remembered how high-level pygame is
compared to using libsdl in C! Pygame has all sorts of great rect
methods and sprite groups and other great features that make it a
real win. So now I'm taking up the fool's errand of the second system.

I've got access to the SAIS code, so I've been trolling through it to
figure out some of the math behind the game object interactions. The
camera math is some of the clever stuff, and I am thinking of making heavy
use of pygame's rect magic to make that happen.

And then I'll make the game AI, situations, ship definitions, and so on
out of modder-supplied python code. I'll have my own default set based on
SAIS, but it should be fun.

But for now I'm working on making the simple tactical system work as a
hollow demo, with space-object-motion and sweeping camerawork. Once that
works in a generalized manner (so that you can raise and lower HUD
elements once that's coded) I'll start making a game of it.

Tue, 13 Dec 2005

Strange Adventures in Infinite Porting

I've got a short list of goofy projects that always seem to come up at
times when I'm feeling nostalgic or hoping to tackle a relatively
uncomplicated programming feat to keep my skills up. I always consider
writing interactive fiction in my new favorite language, or maybe a BBS,
or perhaps that TradeWars clone I've been meaning to try.

I mumbled about TradeWars in IRC one day, and someone mentioned
Diceland. It's a game where you fold these paper octahedra, and the
side that's facing has certain power qualities. You move by pressing
down on green corner dots (causing you to roll over one edge, allowing
you to "walk" across the field) and damage is represented by a red dot
that always takes you to a less powerful face.

On the web site and paper game box, they tout this PC game called
Strange Adventures in Infinite Space. Last night as I was leaving
the ragout to simmer and waiting for E to come home, I decided to head
over to the Web site and drool over screenshots.

So I had a look around and figured it would be another era before Free
Software had a good critical mass of indie game developers to make
shiny little cult hits like this. But I noticed that they claimed to
have a Palm Pilot version or something, so I kept poking around on the
site to see just how cross-platform it really was. Then I hit jackpot.

They released the source code to the game engine under the GPL about a
month ago, in order to open the doors to the modding community that had
sprung up around this game. In addition to this, the authors had ripped
out all the DirectX code and replaced it with SDL, making Linux ports
super easy.

So I quickly pasted this into IRC and set to downloading. After an hour
of ripping out Windowsisms (namely, replacing all the directory-walking
code, and then spending an hour in the debugger before realizing that an
image wasn't being loaded because the coder expected a case-insensitive
filesystem) I had a running demo! It was slow, so this morning I had a
look at how the developers had done the main SDL loop, and realized that
the double-buffering was miscoded. It's a little faster now, but still
not exactly where I'd like it.

You can see my darcs repository at http://zork.net/~nick/sais/ and you
can always just darcs get http://zork.net/~nick/sais/ and join in
with me. I've put in a proper makefile, and I plan to fix the
performance bottlenecks visible in http://zork.net/~nick/sais/gprof.txt
. The big problem is that the developers hand-wrote their own sprite
libraries and just treated DirectX as a dumb buffer to blit pixel data
to. There is a lot of room to replace their code with ordinary SDL
functions, and their sprites with SDL_Surfaces.

In addition to this, I'd love to see if I can get the modders to
contribute some GPL-friendly artwork, and maybe I'll hit the creative
commons sites to find replacement sound samples. I'd love to get Uncle
Pedro to do some pencil-squiggle animations so that we can finally do
the Bread & Cheese game.

Fri, 18 Nov 2005

Testin' and reSTin'

Today just gets better and better. Not only did I hear that my old apartment management agency has agreed to part withy my security deposit (!) but I just found out that tomorrow they'll be showing my favourite holiday film, The Ref.

Mon, 14 Nov 2005

Don't worry. Be happy.

[Back again to finish up after a net hiccup]

I don't recall "mumbling" but Woohoo for reST! I will soon stop polluting the airwaves with HTML. Thanks, Nick, for reducing the LRV content on Laziness to a managable level. I assume that there is a sooper seekrit LRV-only RSS consolidator hidden somewhere (underground). I also need to treat you guys to cre^pes and/or tea and/or Guinness and/or just a good time at some point in the near future.

I just re-read the previous note I posted here and am amused by how much I had wound myself up before my trip back to IL. As it turned out (as it always does) all those worries were groundless. While things didn't go perfectly to plan (e.g. they cancelled my early morning flight from SFO and merged us with the next) it all worked out just fine. I made it in time on Friday to meet the gang in the bar, both to welcome me back and also to say goodbye to one of my ex-associates who had just finished his last day at the old place and was about to follow in my footsteps (or wheel ruts) over to here (now dubbed "The Promised Land"). The IL apartment agency decided not to make a fuss over my leaving (but whether or not I'll see my deposit again is another matter...) and they were fine about my staying in my old place for a few days. I found that they had completely redecorated the apartment in my absence. As well as fixing some long-standing problems they had upped the level of finish so that it looked a good deal better than when I first moved in. It was a little bizarre sleeping on an air mattress under a camping blanket and my coat but that helped give me complete closure on leaving the place. It definitely isn't mine any more. Closing the electricity, gas, phone and bank account went off okay and I had time for Saturday dinner with a friend/ex-colleague and her family followed by Sunday brunch with another friend/ex-colleague and his family unit (they'll also be leaving for The Promised Land early in 2006). On the Monday I snuck back into Ye Olde Salte Mine to meet with my co-authors to work on our current opus. I dodged all in my old department while I was there. IL decided to treat me to the full range of weather - warm and sunny when I first arrived, cool grey and overcast on the Sunday and raining on the Monday. It was good to see the autumnal colours, although the trees outside the window of my new office are changing so I am getting a hint of seasons.

Work at the new place is going very well and I am being eased into my new post. After experiencing first-hand how badly new management can affect a group of people I have no intention of barging in and throwing my weight around. Just to emphasise that point I had two calls from people back in the old department telling me about the latest shenanigans there. Both people were very depressed and one will probably leave. Part of this was triggered by the departure of another of our colleagues. He handed in his notice which apparently infuriated the manager who blew up, called HR and had him escorted from the premises with no opportunity to work notice. The rest of the department were then threatened with the same treatment. People in other departments are horrified about what is going on. The amazing thing is that the new manager has all of the company's senior management completely fooled and is heaping the blame on us who have left. I'm worried about what will happen to all of my friends. On the positive side I have even more confidence in my new company as they are losing a significant rival as they cripple themselves with all of this self-destructive behaviour.

Sun, 30 Oct 2005

Switching to REST

I noticed nutella mumbling about the ability to check the output of his
entries. I only just realized that he was actually entering everything in
HTML, and ignoring the actual plain-text formatting that motd uses. Even his
entries that were just blank unadorned paragraphs were separated with <p> tags
instead of blank lines.

Well I've now switched laziness away from the old "genericwiki" formatter, and
over to reST. This means that shoving HTML code inline will no longer work,
and you'll have to learn reST layout in order to punch in unicode entities or
make non-trivial links. The end result, however, will be to banish those
hideous less-than/greater-than symbols from everyone's posts.

But the bonus is that you can always test by running rest2html on an entry and
checking your result in a browser.