Tuesday, April 21, 2009

Roadmap for JIT

Hello.

First a disclaimer. This post is more about plans for future than current
status. We usually try to write about things that we have done, because
it's much much easier to promise things than to actually make it happen,
but I think it's important enough to have some sort of roadmap.

In recent months we came to the point where the 5th generation of
JIT prototype was working as nice
or even a bit nicer than 1st one back in 2007. Someone might ask "so why
did you spend all this time without going forward?". And indeed, we spend
a lot of time moving sideways, but as posted, we also spent a lot of time
doing some other things, which are important as well.
The main advantage of current JIT incarnation is much much simpler than
the first one. Even I can comprehend it, which is much of an improvement :-)

So, the prototype is working and gives very nice speedups in range of 20-30x
over CPython. We're pretty confident this prototype will work and will
produce fast python interpreter eventually. So we decided that now we'll
work towards changing prototype into something stable and solid. This
might sound easy, but in fact it's not. Having stable assembler backend
and optimizations that keep semantics is not as easy as it might sound.

The current roadmap, as I see it, looks like as following:

Provide a JIT that does not speedup things, but produce assembler without
optimizations turned on, that is correct and able to run CPython's library
tests on a nightly basis.

Introduce simple optimizations, that should make above JIT a bit faster than
CPython. With optimizations disabled JIT is producing incredibly dumb
assembler, which is slower than correspoding C code, even with removal
of interpretation overhead (which is not very surprising).

Backport optimizations from JIT prototype, one by one, keeping an eye
on how they perform and making sure they don't break anything.

Create new optimizations, like speeding up attribute access.

Profit.

This way, we can hopefully provide a working JIT, which gives fast python
interpreter, which is a bit harder than just a nice prototype.

Tell us what you think about this plan.

Cheers,
fijal & others.

Hello.

First a disclaimer. This post is more about plans for future than current
status. We usually try to write about things that we have done, because
it's much much easier to promise things than to actually make it happen,
but I think it's important enough to have some sort of roadmap.

In recent months we came to the point where the 5th generation of
JIT prototype was working as nice
or even a bit nicer than 1st one back in 2007. Someone might ask "so why
did you spend all this time without going forward?". And indeed, we spend
a lot of time moving sideways, but as posted, we also spent a lot of time
doing some other things, which are important as well.
The main advantage of current JIT incarnation is much much simpler than
the first one. Even I can comprehend it, which is much of an improvement :-)

So, the prototype is working and gives very nice speedups in range of 20-30x
over CPython. We're pretty confident this prototype will work and will
produce fast python interpreter eventually. So we decided that now we'll
work towards changing prototype into something stable and solid. This
might sound easy, but in fact it's not. Having stable assembler backend
and optimizations that keep semantics is not as easy as it might sound.

The current roadmap, as I see it, looks like as following:

Provide a JIT that does not speedup things, but produce assembler without
optimizations turned on, that is correct and able to run CPython's library
tests on a nightly basis.

Introduce simple optimizations, that should make above JIT a bit faster than
CPython. With optimizations disabled JIT is producing incredibly dumb
assembler, which is slower than correspoding C code, even with removal
of interpretation overhead (which is not very surprising).

Backport optimizations from JIT prototype, one by one, keeping an eye
on how they perform and making sure they don't break anything.

Create new optimizations, like speeding up attribute access.

Profit.

This way, we can hopefully provide a working JIT, which gives fast python
interpreter, which is a bit harder than just a nice prototype.

Its not obvious from this post what would actually be the difference between the prototype and the final jit with all the prototypes optimisations. So ... it sounds like a lot of work for zero gain. I'm sure there is missing information, like what is actually missing from or wrong with the prototype ( is it in a different language? Prolog?) Without this information its impossible to judge this plan.

I'm extremely excited about seeing this happen. It is an unfortunate fact that the majority of people won't get PyPy until they see a 'big win'. Once they've noticed the big win they will start to see the 'hidden genius'. I'm glad that you are taking such a professional approach to this next phase and look forward to the day when people will start to look give PyPy the attention it deserves (if not for quite the right reason).

responding to rjw: I think the problem was that the prototype was really incomplete, putting all the complexity needed for the rest of the language could be done without removing the optimizations but would make bug finding way harder.

I hope that this could be the only new feature for the next pypy release. Focusing on the JIT might be the best way to attract many more eyes and hands to the project.

herse: that's an approach which is often mentioned, but which does not make sense in PyPy. The JIT is generated from the language spec; whether this spec covers 95% or 100% of Python doesn't change anything. The 95%-versus-100% debate only makes sense at another level, e.g. if we wanted to make PyPy faster without a JIT at all.

"""The JIT is generated from the language spec; whether this spec covers 95% or 100% of Python doesn't change anything."""

i see. the whole pypy idea really sounds awesome to me.

i have another question. your python interpeter is written in rpython so it is supposed to be simpler to work with than the c implementation. but i could imagine that it is incredibly hard to debug problems in pypy-c? doesn't this counterbalance the advantage again?