PyCon 2006 Dallas

My most distinct impression of Dallas was the birds. Two hundred on the
electric wires. Another two hundred swoop in for the party. Others sit on
nearby trees. Four times as many birds as I've ever seen in one
place. All different kinds, all chirping loudly at each other. Now I know
where birds go when they fly south for the winter: to Dallas. Especially to
the intersection of Belt Line Road and the North Dallas Tollway. Around the
intersection stand four or five steel-and-glass skyscrapers. Corporate execs
watch the birds through their windows -- and get pooped on when they
venture outside. But this isn't downtown; it's a suburb, Addison, with a
shopping mall across the tollway. Behind CompUSA is the Marriott hotel where
the Python conference took place.

PyCon was started three years ago as a
low-budget developers' conference. It had always met at the George Washington
University in Washington DC, but attendance has been growing by leaps each
year and had reached the facility's capacity. After a nationwide search, the
Python Software Foundation decided to sign a two-year contract with a Dallas
hotel. That was partly because the Dallas organizers were so enthusiastic and
hard-working, the hotel gave us an excellent deal, and we wanted to see
how many people in the southern US would attend a conference if it were closer
to home. There's no scientific count, but I did meet attendees from Texas,
Utah, Arizona, Nevada, and Ohio, many of whom said they had never been to PyCon
before. Overall attendance was 410, down from 450. Not bad considering the
long move and the fact that many people won't go to Dallas because it's, well,
Dallas. But the break-even point was in the high 300s so it didn't bust the
bank. The layout had all the meeting rooms next to each other and there were
sofas in the hallway, so it was easier to get to events and hold impromptu
discussions than last year. The hotel staff was responsive; there were techs
on-hand to deal with the sound system. The main problem was the flaky wi-fi,
which will hopefully be improved next year (read "better be improved next
year".) A hand goes out to Andrew Kuchling, who proved his ability not only in
coding and documentation ("What's New in Python 2.x?") but also in
conference organizing. This was his first year running PyCon, and he got
things running remarkably smoothly.

There seemed to be more international attendees this year. I met people
from the UK, Ireland, Germany, the Netherlands, Sweden, Japan, Argentina, and a
German guy living in China. This is in spite of the fact that EuroPython and
Python UK are now well established, and the main pycon.org site is now an umbrella covering
both.

Here's the conference schedule. Several of the
talks were audio- or video-recorded and will be available here.

Keynotes

Guido Note 1
delivered two keynotes. One was his usual "State of the Python Universe".
The other was a look backward at Python's origins. The latter covered
territory similar to this 2003 interview, which explains
how Guido created Python to try out some language ideas and improve on ABC,
a language he'd had both good and bad experiences with. He also explained
why Python doesn't have type declarations: "Declarations exist to slow down
the programmer." There you have it.

The state of the Python universe has three aspects: community activity,
changes coming in Python 2.5, and changes coming in Python 3.0. There has been
a lot of activity the past year:

The Python
source is now hosted at python.org in
Subversion rather than in SourceForge CVS.

The
Buildbot, which was written in
Python, now watches over the Python source. Whenever somebody checks in a
patch, the bot tells several volunteer computers on different architectures to
download and compile Python and report if anything has broken. "The guilty
developer can be identified and harassed without human intervention." The
current results are visible on the
Web.

EasyInstall
and eggs are
getting some real-world use, especially with the
TurboGears project, which depends on
some ten third-party packages. ("TurboGears would not have been possible
without eggs," says its main developer.)

An .egg is like a Java .jar file: a package that knows its
version, what it depends on, and what optional services it can provide to other
packages or take from other packages. This is similar to .rpm and .deb but is
OS-neutral. It is expected that Linux package managers will eventually use
eggs for Python packages. Eggs can be installed as directories or zip files.
easy_install.py is a convenient command-line tool to download and
install Python packages (eggs or tarballs) in one step. It will get the
tarball from the Python Cheese ShopNote 2 (formerly known as the Python Package Index),
or scrape the Cheese Shop webpage for the download URL. You can also
provide the tarball directly, or another URL to scrape. Regardless of whether
the original was an egg or a simple tarball, EasyInstall will install it as an
egg, taking care of the *.pth magic needed for eggs.

Waitress: Well, there's egg and bacon; egg sausage and bacon; egg
and spam; egg bacon sausage and spam; spam bacon sausage and spam; spam egg
spam spam bacon and spam....Wife: Have you got anything without spam?Waitress: Well, there's spam egg sausage and spam, that's not got
much spam in it.Wife: Could you do the egg bacon spam and sausage without the spam
then?

Python 2.5 changes

The first alpha is expected May 6; the final by September 30. Python 2.4.3
will be released in April, followed by 2.4.4, the last of the 2.4 series.
What's
New in Python 2.5 is unfinished but explains the changes better than I
can.

The most sweeping changes are the generator enhancements
(PEP 342)
and the new with keyword
(PEP 343). These allow
you to write coroutines and safe blocks.
"Safe" means you can guarantee a file will be closed or a lock released
without littering your code with try/finally stanzas. The
methodology is difficult to understand unless you have a Computer Science
degree, but the standard library will include helper functions like the
following:

The same pattern works for blocking signals, pushing/popping the locale
or decimal precision, etc.

Coroutines are generators that you can inject data into at runtime.
Presumably the data will be used to calculate the next yield
value. This not only models a series of request/response cycles, but it
also promises to radically simplify asynchronous programming, making
Twisted much more accessible. Twisted
uses callbacks to avoid blocking, and that requires you to split your code into
many more functions than normal. But with coroutines those "many functions"
become a single generator.

New functions any() and
all(), which return true if any or all of their arguments are
true.

Curried functions via
functional.partial.

Dictionaries that accumulate all values for a key
in lists (collections.defaultdict).

Several third-party packages that will will hopefully be merged into the
standard library:

Setuptools (and thus
easy_install.py)

WSGIRef (a protocol for web application
servers)

cElementTree (a pythonic XML DOM)

ctypes (call a C function directly from
Python)

Guido resisted ctypes for a long time because it "provides new
ways to make a Python program dump core" by exposing it to arbitrary C bugs.
Skip Montanaro responded by listing several ways you can already make Python
dump core (try these at
home), and it was decided that ctypes wasn't any worse than
those.

Python 3.0 changes

Now that Guido is employed at Google and can spend 50% of his paid time
hacking Python, 3.0 doesn't have to wait until sometime after his 5-year-old
son Orlijn graduates college. The planned changes for 3.0 are listed in PEP 3000. Guido
highlighted the string/unicode issue: the str type will be
Unicode, and a new bytes type will be for arbitrary byte
arrays.

There's one late-breaking change, or rather a non-change: the unloved
lambda will remain as-is forever. lambda creates
anonymous functions from expressions. Attempts to abolish it were undone by
use cases that required syntax that's arguably worse than lambda,
and Ruby-style anonymous code blocks didn't fare any better. Programmers who
overuse lambda should still be shot, however.

BitTorrent

The other keynotes were on Plone and
BitTorrent. I missed the Plone talk,
but for BitTorrent Steve Holden interviewed Bram Cohen, BitTorrent's creator.
Bram talked about hacking while slacking (he wrote most of his code while
intentionally unemployed and living on savings), his new job as the owner of
Bittorrent Inc (not much time to code), why he chose Python for BitTorrent,
why BitTorrent doesn't make you into a carrier of malware (you won't be
uploading anything you didn't specifically request to download), Pascal, and
many other topics.

I made a wonderful faux pas the first evening at dinner when I sat
next to Bram, unaware he was going to speak. I asked what he did, and he said
he invented BitTorrent. I didn't remember what that was and kept thinking of
Linux's former version control system, whose name I couldn't remember. Yet
the term "torrent file" kept crossing my brain, clearly a configuration file
and not related to the Linux kernel. Finally I remembered, "BitTorrent is that
distributed file download thing, right?" Bram said yes. So I asked the guys
across the table, "What was the name of Linux's former version control system?"
They said, "BitKeeper". Ah ha, no wonder I got them confused. I
thought no more about it, then Bram ended up mentioning BitKeeper several times
during his keynote, usually in terms of how bad it is. He talked about
BitKeeper vs. git (Linux's new version control system), git vs. other things, and
then about BitTorrent vs. Avalanche. Avalanche is a distributed file
download system from Microsoft, which Bram called vaporware in
his blog, stirring up a lot of controversy (including a newspaper article in
Australia).

For those who think BitTorrent is all about illegally downloading
copyrighted songs and movies, Bram points to SXSW, a music
and film festival which will be using BitTorrent to distribute its
performances. "The Problem with Publishing: More customers require more
bandwidth. The BitTorrent Solution: Users cooperate in the distribution."
Other articles point out that a BitTorrent client normally interacts with
30-50 peers, reducing the strain on the original server by thirtyfold.

Bram also warned people to download BitTorrent directly from
www.bittorrent.com and not from some
random Google search. Shady operators are selling scams that claim to be
BitTorrent but contain spyware or viruses. The real BitTorrent is free to
download, and is Open Source under a Jabber-like license. The company does
accept voluntary donations, however, if you really want to give them money.

Session talks

The rest of PyCon was session talks, tutorials, lightning talks, Open Space,
sprints, and informal "hallway discussions". The most interesting talks I
saw or wanted to see were:

Nokia has released a version of Python for its
S60 mobile phone which includes Bluetooth and mobile Internet. The slides
showed it displaying stats from the GSM towers, weather reports, a stock market
graph, an online dictionary, and a stupid guitar tuner. There are Python
libraries to access the phone's battery usage, to dial a number, take a
picture, manage your inbox and address book and calendar, and display arbitrary
graphics and sound. You can write programs using an emulator and download the
working programs to the phone via Bluetooth. (Note: the URL points to a slide
show. Click anywhere to go to the next page. There appears to be no way to go
backward.)

Dabo is an application development framework
inspired by Visual Basic. It provides a high-level API for GUI design and
SQL database viewers. The database viewer is something I've wanted for years: a
simple way to make
CRUD grids and
data-entry forms similar to Microsoft Access. Only MySQL, PostgreSQL, and
Firebird are supported so far, but others are on the way. The GUI API is so
elegant that the wxPython developers themselves have started using it rather
than their own API, which is a thin wrapper around the wxWidgets C++
library.

State-of-the-Art Python IDEs

A survey of several Python IDEs, based on
research
done by the Utah Python Users' Group.

Building Pluggable Software with Eggs

A tutorial on Python's new packaging format.

Docutils Developers Tutorial: Architecture, Extending, and Embedding

Docutils is the project name for ReStructured Text
(ReST), a wiki-like markup format. Several talks mentioned using
Docutils, and several Python projects have adopted it for documentation, data
entry, etc. One talk explained how to make slides with it. Another speaker
didn't make slides and had to download and learn Docutils the night before the
talk, but he was able to get his slides done anyway.

ReST includes a construct called
field lists
that look like email headers; they represent virtual database
records. ReST does not define the database or say what kind of record it is,
only that it's a record. Nabu is the first application I've seen to fully
exploit this potential. Nabu helps you extract relevant records from all
documents into a central database, which can then be queried or used in an
application. You'll have to decide beforehand
which fields a "contact record" or "URL bookmark" should contain, and how
you'll recognize one when you see it. Nabu's author Martin Blais keeps an
editor open for every task he does, to record random notes, TODO plans,
contacts, and URL bookmarks. The random notes are unstructured ReST sections;
the contacts are field lists scattered throughout the document. He also has a
file for each trip he's been on, containing random observations, restaurants to
remember (or forget), etc. Nabu is also suitable for blog entries, book lists,
calendar events, class notes, etc. One constraint is you must put "something
else" between adjacent records so ReST knows where each record ends. Putting
multiple records in a
bullet list
satisfies this constraint.

Web frameworks

"Python has more web application frameworks than
keywords," it has been rightly observed. A few people go further and say,
"You're not a real Python programmer until you've designed your own web
framework." (Hint: subclass BaseHTTPServer.HTTPServer in the
standard library; it's easy!) So it's not surprising that PyCon had three
talks on TurboGears, two on Django, three on Zope (plus several BoFs), and a
couple on new small frameworks. All the discussion last year about framework
integration has made them more interoperable, but it has not cut the number of
frameworks. If anything, they've multiplied as people write experimental
frameworks to test design ideas. Supposedly there's a battle between
TurboGears and Django for overall dominance, but the developers aren't
competing, they just have different interests. Jacob Kaplan-Moss (Django
developer) and I (TurboGears developer) ran the Lightning Talks together, and
we both left the room alive. Some people work on multiple frameworks, hoping
the
holy grail
will eventually emerge. Much of the work focuses on WSGI and Paste, which help tie diverse components
using different frameworks together into a single application. Some work
focuses on AJAX, which is what makes Gmail and Google Maps so responsive and is
slowly spreading to other sites.

Lightning talks

Lightning talks are like movie shorts. If you don't like one, it's over in
five minutes. They are done one after the other in hour-long sessions. Some
people give lightning talks to introduce a brand-new project, others to focus
on a specialized topic, and others to make the audience laugh. This year there
were seventeen speakers for ten slots, so we added a second hour the next day.
But in the same way that adding a new freeway lane encourages people to drive
more, the number of excess speakers grew rather than shrank. We ended up with
thirty speakers for twenty slots, and there would have been more if I hadn't
closed the waiting list. The audience doesn't get bored and keeps coming back,
so next year we'll try adding a third hour and see if we can find the
saturation point. Some of the highlights were:

Facundo Batista talked about Python use in
Argentina and the PyAr users
group. There are a hundred members throughout the country, and around fifteen
that gather for a meeting in Buenos Aires.

Internet Censorship in China

Philipp von Weitershausen gave a quick overview
of what it's like to use the Internet in China. There are 120 million users,
with 20 million more added every year. Chinese sites respond much more slowly
than international sites. Some 50,000 officials monitor what sites people
visit. Wikipedia, the BBC, and VOA are blocked completely due to undesirable
political content. Other sites go up and down, and the error keeps changing.
Sometimes it's a timeout, other times an error message. Encrypted connections
timeout. How do people feel about not having access to Wikipedia? Only 2% of
Chinese Internet users know it exists, so most don't know what they're missing
or think it's a big deal.

How to Replace Yourself with a Small Bot

Ben Collins-Sussman had too much time on his
hands and wrote an IRC bot that masqueraded as himself. He put it on a channel
and waited to see how long until people noticed. The bot had five or so canned
remarks like "Did you check the FAQ?", which it recited at random whenever
anybody addressed him. Hmm, maybe The Answer Gang
here at the Gazette needs one of those. The experiment didn't succeed too
well because the bot said wildly inappropriate things at the wrong
time.

[ I don't understand what you mean by "didn't
succeed", Mike - seems to me that wildly inappropriate responses to a
question is perfectly normal IRC behavior... so how could anyone tell? --
Ben ]

There were other good talks too but since I was coordinating I couldn't
devote as much attention to them as I would have liked.

Sprints

I attended the TurboGears sprint and worked on Docudo, a wiki-like beast for
software documentation, which groups pages according to software version and
arbitrary category and distinguishes between officially-blessed pages and
user-contributed unofficial pages. We wrote up a spec and started an
implementation based on the
20-Minute Wiki Tutorial.
This version will store pages in Subversion as XHTML documents, with
Subversion properties for the category and status, and use
TinyMCE for editing. TinyMCE
looks like a desktop editor complete with toolbars, but is implemented in
Javascript. We've got parts of all these tasks done in a pre-alpha application
that sort of works sometimes.

Other TurboGears fans worked on speeding up Kid templates, adding unittests,
improving compatibility with WSGI middleware and Paste, using our configuration
system to configure middleware, and replacing CherryPy with RhubarbTart and
SQLObject with SQLAlchemy. Don't get a heart attack about the last two: they
are just experimental now, won't be done till after TurboGears 1.0, and will
require a smooth migration path for existing applications. We had a variety of
skill levels in our sprint, and some with lesser skills mainly watched to learn
some programming techniques.

There were ten people in the TurboGears sprint and fifty sprinters total.
Other sprinters worked on Zope, Django, Docutils, the Python core, and a few
other projects.

The sprint was valuable to me, even though I'm not fully committed to
TurboGears, because I'm starting to write TurboGears applications at work:
it was good to write an application with developers who know more than I do
about it. That way they can say, "Don't do that, that's stupid, that's not the
TG way." I would have liked to work on the RhubarbTart integration but I had to
go with what's more practical for me in the short term. So sprinting is a
two-way activity: it benefits the project, and it also benefits you. And it
plants the seeds for future contributions you might make throughout the
year.

Impressions of Dallas

Dallas was not the transportation wasteland I feared (ahem Oklahoma City,
Raleigh, Charlotte...) but it did take 2-3 hours to get to PyCon without a car,
and that includes taking a U-shaped path around most of Dallas. An airport
shuttle van goes from the sprawling DFW campus to the south parking lot a mile
away. From there another airport shuttle goes to the American Airlines
headquarters, an apartment building (!), and finally the Dallas - Fort Worth
commuter train. That's three miles or thirty minutes just to get out of the
airport. The train goes hourly till 10:30pm, but not on Sundays. It cost
$4.50 for an all-day rail/bus pass. The train pokes along at a leisurely pace,
past flat green fields and worn-down industrial complexes, with a few housing
developments scattered incongruously among the warehouses. Freight trains
carried cylindrical cars labelled "Corn Syrup" and "Corn Sweetener". Good
thing I wasn't near the cars with a hatchet; I feel about corn syrup the way
some people feel about abortion. The train stopped in downtown Dallas at an
open section of track euphemistically called "Union Station". I transferred to
the light rail (blue line) going north. This train was speedy, going 55 mph
underground and 40 mph above, with stops a mile apart. Not the slowpoke things
you find in San Jose and Portland; this train means business. The companies
along the way seem to be mostly chain stores. At Arapaho Station (pronounced like a
rapper singing, "Ah RAP a ho!") in the suburb of Richardson, I transferred to bus
400, which goes twice an hour. A kind soul on the train helped me decide which
bus to catch. The bus travels west half an hour along Belt Line Road, a
six-lane expressway. It crosses other six-lane expressways every twenty
blocks. Dallas has quite the automobile capacity. We're going through a black
neighborhood now. The driver thinks the Marriott is a mile away and I
should have gotten another bus, but the hotel map says it's at Belt Line Road
and the North Dallas Tollway. When we approach the intersection with the
birds, all is explained. The road the hotel is named after goes behind the
hotel and curves, meeting the tollway. So the map was right.

[ A note from my own experience in Dallas, where I
teach classes in the area described above: a shuttle from DFW is ~$20 for
door-to-door service, and takes less than a half an hour. -- Ben ]

Around the hotel is mostly expense-account restaurants for the executive
crowd. We didn't find a grocery store anywhere. So I learned to eat big at
meals because there wouldn't be any food until the next one. There was a mall
nearby and a drugstore, for all your non-food shopping.

The weather was... just like home (Seattle). Drizzle one day, heavy rain
the next, clear the day after. Same sky but ten degrees warmer. Then the
temperature almost doubled to 80 degrees (24C) for three days. In February!
I would have been shocked but I've encountered that phenomenon in
California a few times.

Saturday evening a local Pythoneer took two of us to downtown Dallas.
Downtown has several blocks of old brick buildings converted to loft apartments
and bars and art galleries, and a couple coffeehouses and a thrift shop.
Another feature is the parking lot wavers. I swear, every parking lot had a
person waving at the entrance trying to entice drivers. It's 10pm, you'd think
the lot attendants would have gone home. Especially since there were plenty of
metered spaces on the street for cheaper. There weren't many cars around:
there were almost as many parking lots as cars! It was a bit like the Twilight
Zone: all these venues and not many people. We went to Café Brazil,
which is as Brazilian as the
movie. In other
words, not at all.

PyCon will be in Dallas next year around February-April, so come watch the
birds. The following year it might be anywhere in the US.

Footnotes

2
The name "Cheese Shop" comes from a Monty Python
skit.
The Cheese Shop was formerly called the Python Package Index (PyPI), but was
renamed because PyPI was confused with
PyPy, a Python interpreter written in
Python. They are both pronounced "pie-pie", and the attempt last year to get
people to call PyPI "pippy" failed. Some people don't like the term "Cheese
Shop" because it doesn't sell any cheese. But the shop in the skit didn't
either.

Mike is a Contributing Editor at Linux Gazette. He has been a
Linux enthusiast since 1991, a Debian user since 1995, and now Gentoo.
His favorite tool for programming is Python. Non-computer interests include
martial arts, wrestling, ska and oi! and ambient music, and the international
language Esperanto. He's been known to listen to Dvorak, Schubert,
Mendelssohn, and Khachaturian too.