Wednesday, April 22, 2009

Shed Skin 0.1.1

I have just released version 0.1.1 of Shed Skin, an experimental (restricted) Python-to-C++ compiler. It comes with 5 new example programs (for a total of 35 example programs, at over 10,000 lines) and several important improvements/bug fixes. See here for the full changelog.

The most interesting new example is minilight, an elegant raytracer (more precisely, a global illumination renderer) that utilizes triangle primitives and an octree spatial index. As shown on the minilight homepage, it becomes up to 100 times faster.

The fifth new example, an interactive circle packing program, is especially nice, because it is, well, interactive, but also because it shows how easy it is to generate an extension module with Shed Skin and use this in a larger program (a Pygame application in this case).

The biggest improvement (if you can call it this) for this release was to drop support for generic types. This simply means that functions that can be called with different kinds of types of objects for a single argument are no longer supported (unless these types have a common base-class, of course). Similarly, generic datastructures are no longer supported. Dropping support for generic types made the compiler core a lot simpler (and 10% smaller!), and removed the need for the -i command-line option.

Seeing the compiler core shrink so much, I was inspired to further refactor several messy areas of the compiler for readability (most notably in infer.py). This means the compiler core has also become a lot more readable/hackable with this release. I hope to continue this work of refactoring (and adding docstrings) with each release from now on, and invite anyone to help out here.

In general, I would really like to receive more help. For example, if someone could take over maintainership of the Windows version (and possible upgrade the MingW distribution that is packaged with it), that would be great. A suggestion on the coding side might be to add keyword support to generated extension modules (see extmod.py), or to optimize some builtins (for example, string/list slicing, see lib/builtin.?pp). But I would also be happy to receive more interesting test cases (most come from a single person at the moment) and/or more bug reports (which I get far too few of.)

30 comments:

I don't actively use Shed Skin but I follow its development, because it sounds like it will come in handy some time. Even if it doesn't, I'm sure there will be many useful advances in speeding up Python stemming from your efforts.

thanks! this looks correct - one test doesn't work because the 'signal' module currently fails under Windows, and the other tests fail because Shedskin still acts like Python 2.5, rather than 2.6, which causes some subtle differences.

wow: core loop gets down to about 6 seconds (from the original 42!) with -w -b. no random numbers, so no -r needed. with g++ 4.0.1, no big performance gain from -fprofile-generate/use. looking into gprof2dot right now!

The latest version of gcc seems to have caused an issue with Shedskin:

/usr/lib/shedskin-0.1/lib/builtin.cpp:684: error: ‘uint32_t’ does not name a type /usr/lib/shedskin-0.1/lib/builtin.cpp: In member function ‘virtual int __shedskin__::str::__hash__()’: /usr/lib/shedskin-0.1/lib/builtin.cpp:734: error: ‘SuperFastHash’ was not declared in this scope

What I meant to say, was that I have a Python script that I want to compile with Shedskin. However, with Shedskin (svn) and GCC 4.4 (default for Arch Linux), thousands of lines of errors appear when GCC tries to compile the output from Shedskin.

but note that you can already combine GUI libraries and Shedskin-compiled code, by generating an extension module (shedskin -e, see the tutorial). this way, you can use unrestricted Python and arbitrary libraries in the 'main' program, while speeding up some part that really needs a speedup.

This program looks great. As soon as I'm finished work, I will try it on my python implementation of the Dijkstra algorithm. I also have written the same algorithm in C++, so I'm looking forward to test if it works, and if so, see how fast the python to C++-translated code is compared to mine C++ implementation.

If it works well, you can add the Dijkstra solving algorithm to your examples.