Introducing Pyston: an upcoming, JIT-based Python implementation

Hello everyone, I’m very excited to announce Pyston, a new open-source implementation of Python, currently under development at Dropbox. The goal of the project is to produce a high-performance Python implementation that can push Python into domains dominated by traditional systems languages like C++.

Here at Dropbox, we love Python and try to use it for as much as we can. As we scale and the problems we tackle grow, though, we’re starting to find that hitting our performance targets can sometimes become prohibitively difficult when staying on Python. Sometimes, it can be less work to do a rewrite in another language. I personally love Python, and it pains me every time we decide to rewrite something, so I wanted to do something about it. After some abandoned experiments with static compilation, we looked around and saw how successfully JIT techniques are being applied in the JavaScript space: Chrome’s V8 engine, in particular, has greatly pushed the status quo of JavaScript performance. Our hope is that by using similar techniques, we can achieve similar performance improvements for Python.

Pyston is still in the earliest stages and is not ready for use, but we’re hopeful that by announcing it early in its lifecycle and open-sourcing the code, we can collaborate with the Python and JIT communities throughout its development. There’s only room for so much detail in this blog post, but we wanted to talk about why we think we need a new Python implementation, and go into a little bit of how Pyston works.

Why a new implementation

There are already a number of Python implementations using JIT techniques, often in sophisticated ways. PyPy has achieved impressive performance with its tracing JIT; Jython and IronPython are both built on top of mature VMs with extensive JIT support. So why do we think it’s worth starting a new implementation?

In short, it’s because we think the most promising techniques are incompatible with existing implementations. For instance, the JavaScript world has switched from tracing JITs to method-at-a-time JITs, due to the compelling performance benefits. Whether or not the same performance advantage holds for Python is an open question, but since the two approaches are fundamentally incompatible, the only way to start answering the question is to build a new method-at-a-time JIT.

The downside of starting from scratch is, unsurprisingly, that creating a new language implementation is an enormous task. Luckily, tools are starting to come out that can help with this process; in particular, Pyston is built on top of LLVM, which lets us achieve top-tier code generation quality without having to deal with the details ourselves. Nonetheless, a new Python implementation is a huge undertaking, and Pyston will not be ready for use soon.

How it works

At a high level, Pyston takes parsed Python code and transforms it to the LLVM intermediate representation (IR). The IR is then run through the LLVM optimizer and passed off to the LLVM JIT engine, resulting in executable machine code. LLVM contains a large number of optimization passes and mechanisms for easily adding more, which can lead to very fast code.

The problem, though, is that LLVM can’t reason about Python code, because all the low-level behavior is hidden behind the type dispatching you have to do in any dynamic language. To handle this, Pyston employs type speculation: it is typically impossible to prove that a variable will have a specific type, but Pyston can often predict with some certainty what the type of an object can be. Once a prediction is made, Pyston will verify the prediction at runtime, branching between a fast path where the prediction holds, and a slow path where it doesn’t.

Pyston also includes other modern techniques such as hidden classes for fast attribute lookups and inline caches for fast method calls. You can find more technical details on the Github page, along with a separate blog post that goes into more technical detail.

Current state

Pyston is still in its infancy and right now only supports a minimal subset of the Python language. It’s not quite fair to state benchmark numbers, since 1) Pyston doesn’t support a large enough set of benchmarks to be representative, and 2) Pyston doesn’t support all runtime features (including ones that might introduce slowdowns), so it’s not a true apples-to-apples comparison. With those caveats, Pyston generally is able to beat CPython’s performance, but still lags behind PyPy.

The code has been released on Github under the Apache 2.0 license, along with a growing amount of technical documentation. There’s a lot of work to be done, and we’re looking to grow the team: if this kind of thing interests you, please apply!

Stay tuned for more updates as the project progresses. If you’d like to subscribe to our announcement mailing list, you can do so here.

Do you really feel like your claim that JS engines moved away from tracing JITs is actually accurate in light of the fact that only Firefox (TraceMonkey) ever used a tracing JIT, and it had a number of well documented flaws; and further, perhaps the most impressive dynamic language JIT, LuaJIT2 is still a tracing JIT?

Yes, probably the reason for this claim: “Another point of differentiation is the planned use of a conservative garbage collector to support extension modules efficiently. ” This may be a hugely significant engineering decision.

PyPy has its own drawbacks and cons. It will be interesting to have a different JIT with its own set of pros and cons. Even if it doesn’t turn out to be faster, it may turn out to suit some people’s use cases more.

I don’t see anything wrong with experimenting with some other ideas (for science!), although I think that long term it would be better to combine forces rather than running two separate projects unless there are two fundamentally different use cases that each one is targeting (which right now I don’t think there is).

I’d love to hear more about your issues with PyPy, it sounds like you wrote off PyPy simply because you don’t understand why it works so well. Not to mention that this is mostly a re-hash of stuff found in unladen-swallow.

I mean, if your end goal is to write another Python, sure go for it. But it really sounds like you people haven’t done their research. I see nothing to write home about.

You do know Guido van Rossum works for DropBox now right? If you’re not sure who that is then Google him. He has a nice large Wikipedia page saying that he “is best known as the author of the Python programming language”

This seem to be a completely different approach to LLVM support for pypy (which the pypy team have attempted a few times, so far without success)… the gamble/interesting question is which design choice will lead to better perf in the long run… Dropbox are betting on method-at-a-time JITs!

That addresses “why not PyPy,” but they totally fail to address “how is this different from Unladen Swallow,” which took exactly this approach just a few years ago and was ultimately unable to achieve much of a performance win while maintaining full compatibility. Among other things, LLVM is a great static compiler, but very slow and high-overhead for a JIT. With dynamic languages, where type speculation often requires regenerating code relatively frequently, this can be an especially big issue.

I agree 100% – something like this could be the thing that creates the much needed interest in 3.x. I would bet all of my money (so not too much) that Python 3 would be a huge success if it had all the current features and libs, and was also 3-5 times as performant, in terms of speed and memory usage.

I’m aware, as all the naysayers will inexorably point out, that this is a large undertaking and will likely fail… just like every great achievement in history. In other words, bravo to Dropbox for funding this research when they could have just as easily dropped Python for Java, like Twitter did with Ruby, and bravo to you for giving this a shot. I hope it is a huge success. I, for one, have been whining about how Python needs more horsepower to stay afloat with the advent of Go, Node.js, et al, and I’m excited to see somebody working to add horsepower.

I remember reading that PyPy cannot ‘compile’ Python files, i.e. you have to wait each time a Python script. Numba, a LLVM-based JIT compiler for numeric functions will have such an option (I remember using Numba a year ago, and I had to wait 10-15 seconds each time I ran my script before it actually would start doing things and not compile). Will Pyston have the same option?

There is another platform you could potentially build upon. The Rubinius project has built a Ruby implementation on top of LLVM (per method JIT). Interestingly, it has also been designed as a platform for others to build their own *dynamic* languages on top of this runtime.

Take a look at http://rubini.us/projects/ for other languages that have been built on top of the Rubinius runtime. Note that there is a (somewhat old, probably bit-rot) Python implementation called Typhon.

Rubinius has some pretty good corporate support in the form of Enova (a Chicago-based firm). Perhaps there is an opportunity here for DropBox to jump on board a project that has very similar goals and a similar approach with LLVM.

Some friends of mine who were working on a similar effort also looked at Rubinius and the Shotgun VM, and concluded that if you really want to be pushing performance, VMs really have to become quite tailored for the language they’re running.

Interesting history. If they looked at the “shotgun” VM then they were looking at Rubinius probably 3+ years ago. Shotgun was replaced by the current incarnation which is built to utilize LLVM for compiling and JIT.

It’s probably worth another look now. I love re-inventing the wheel (I do it regularly) but this is an opportunity to short-cut a proof-of-concept. If it pans out, then the decision can be made to start from scratch or continue forward with the rubinius platform.

I’m wondering if the way you’re planning to handle extension models is compatible with using the existing scipy? The inability to do that has always seemed to me to be a major drawback of PyPy’s approach. If you can use scipy, I’ll be very eager to try Pyston for my statistical work.

Talking and comparing to node.js, makes me recall how V8 was originally marketed as being able to support not just JavaScript, but “any” other language. And of course JavaScript is much trickier beast to run than Python is. Did you look into V8 as part of your research?

Why not join forces with Continuum.io and the Numba team? They already have an LLVM based JIT for Python running. It’s already so advanced that it can parallelise for loops on multiple cores, and even make code that runs on the GPU.

Just being curious… Unladen swallow seems to have gone to great lengths before google decided it was not worth for them to continue on it, although it seems they did have some (minor) performance improvements (http://legacy.python.org/dev/peps/pep-3146/) and from the retrospect what was missing was engineering cycles to do other optimizations. Have you considered taking over from where they stopped? (the main thing is that unfortunately PyPy will probably never be really be used because the approach is not compatible with extension modules, and from your proposal it seems that the same thing will happen…)

It would be better to start with Numba than the Unladen Swallow. Numba works great already, and often gives 100-1000 times speed boosts over Python for numeric or algorithmic code. http://numba.pydata.org

Correct, but when Numba can infer the types it will be as fast as C. So for algorithmic code you often get extreme speedup relative to Python. But Numba is still under development, and might get these kinds optimisations if more people care to contribute.

Possibly… although numba is very focused on making one algorithmic function very fast (so, not sure how suitable it’d be for a generic speedup with tracing — which is what unladen swallow targeted), but if the numba team would also be interested in changing their focus to tracing the whole vm it could be interesting.

Anyways, I’m just seeing this from the outside and if I have to choose between something that works without being really python compatible (such as pypy) or something python-compatible as numba, I’d surely choose numba (but really, for me it’s no real problem going to c/c++ to speed that 0.1% of the cases that deserve it — although if a real jit did its work that could go to something like 0.01%, but if something as unladen swallow made things 2x faster overall, it’d already be worth it — as long as extension modules — and things such as numba — keep working).

This may sound a bit silly but humour this question: if this is such large undertaking why not target Python3 directly ? Surely Python 3 will have very good adoption by the time Pyston is production ready no ?

As an enthusiast of PyPy, I’m also excited about Pyston. I’m only curious about the decision to implement it around v2.7. Assuming Guido’s blessing of the project and taking into account the announced and imminent end of 2.7 era, it doesn’t seem in sync with the future of Python. As I see it, the future resides in 3.X and newer versions and advocating them so people migrate and Python can continue evolving.
I only can come up with thinking that it has something to do with your current codebase at Dropbox. If that is so, is it really a good move to bet a herculean effort on a soon to be unsupported version instead of migrating to Py3 already?

Too many assume Python 3.x will take off and want to belittle those who stay on 2.7. But nobody at this time, really knows for sure if Python 3.x will take off. So far Python 3.x is not off to a good start as it’s adoption rates are so low and from my personal experience I don’t even know one project that use Python 3.x in production. Even PyPy get used far more in production than Python 3.x and yet many consider it a failure.

So far no version of Python 3 offers a large enough tangible benefit to even make a small subset of existing production systems make the switch. How ever many do make the switch to PyPy as it offers a nice tangible benefit to many projects.

Right now from a practical point of view it is better to write Python 2.7 code while keeping in mind the changes that were made to Python 3.x so that you can write code that is mostly compatible with Python 3.x. Then if and when Python 3.x takes off you will be prepared to make the switch. Sometimes it’s just better to play it safe than take a chance with hype.

While I agree that Python 3 adoption has being disappointing, I don’t think moving to it can be labeled as ‘taking a change with hype’. The decision on ending Python 2.X support has been made and Python 3 is the way to go from now on.
Python 3 first releases were more of “cleaning up” the language than offering real advantages, but as of 3.4 the scenario is starting to change.
Playing safe and staying with Python 2.7 is understandable, my point though is that all further development will be useless and we’ll be doomed to stay in a deprecated version of the language if nobody is willing to make the jump. For Dropbox it might make sense to stay on 2.7, but for the Python language itself it would be great to have something so desirable as a JIT compiler to drive conversions.

I don’t agree that ending Python 2.x support automatically makes Python 3.x the way to go from now on. It depends heavily on whether or not Python 3 starts to pick up a sizable share of the Python market. If it is unable to do so, the Python 2.x line will be continued.

Now I would like to see Python 3 take off but I’m going to be practical about my use of Python 2 vs Python 3. Right now none of my customers want Python 3 and I’m not willing to take the risks involved moving my customers code over to Python 3 when there is essentially no benefit in do so and in fact it comes with a negative benefit of extra costs.

Coding with Python 2.7, with 3.x syntax when possible, plus of course lots of automated testing, minimizes the possible future work of migrating a code base over to 3.x, if and when it becomes necessary to do so. I believe this is the approach that most are taking and is why there is so little adoption to Python 3.x.

If Python 3.x adoption is going to pick up it is going to need to add some killer features. It needs to add a jit and or add support for multi cores or some other feature that has a huge benefit. At this point in time I don’t see any of these features being added to the CPython 3.x line. Plus at this time, having a jit is almost an expected feature that a dynamic language requires.

Many CPython core devs have expressed that don’t want to add the complexity of a jit to the CPython code base. They are also the ones who forgot about a very import part of the Zen of Python when Python 3 was developed. The forgot about “practicality beats purity”. A large number of decisions for Python 3 was for purity and not practicality.

You see they placed the language as the most important aspect of Python when in reality the most important aspect is its ecosystem. So now we have a more pure Python at the expense of a fractured ecosystem.

The lesson learned here is if you going to make backwards incompatible changes you better make sure there are benefits that out-way the costs.

Ohh… I was so excited about Pyston, but this line “currently targets Python 2.7” ruined all expectations. Developers stuck with Py2 as they think there is no notable improvements in Py3. I’m not agreed with it, and moved to 3 long time ago. But JIT compilation could be a huge bonus could stimulate developers to upgrade. But now I just can’t understand it’s REALLY hard to update that old code at Dropbox? As always business kills all progress and in Python it feels much more painfull… How can you tell about future of python and working on enhancements for old version? This is so sad, legacy code rules the future of the language. It is fail…

If it was easier to upgrade, it wouldn’t be a dreaded task for those with legacy code. 3 is better, but all the same tasks can be done in 2, so if there’s no dealbreaker in 2 and it’s not straightforward to jump to 3, it ain’t gonna happen. It’s not business’s fault we chose to leave behind users of 2, insisting they put forward all the effort to upgrade.

All the same tasks can be done in any turing complete language so why bother? Most of the popular languages have backward incompatible changes in major versions but there is no such circus as in python community. It is ridiculous. Or you want revolution in every update? What do you want from new version? New APIs and modules? There are a bunch of them. Performance improvements? GIL in Py3 works much better, and JIT compilation could be that improvement.

Go on, follow the perl history. lets make a Perl6 from Py3 and left them all in a same oblivion. It is clear now – Dropbox it is the worst company for Guido to work. As any other company addicted to legacy code. Thankfully “all the same tasks” guys not maintaining popular libraries and most of them works well with Py3.

Sorry the WOS website does not prove that the majority of users have all there dependencies available in Python 3.x. It just indicates that most users have at least some of there dependencies available.

To cover the majority of users that web site would likely have to have at least the top 800 projects. The 800 number is just a gut feel and someone would have to look at the downloads figure of all projects in pypi to determine ow many projects should be included before we get to the projects that are part of the long tail of projects that aren’t used that often.

I know for myself I regular use more than 100 projects that are not on WOS. Of course I also use a large percentage of the ones listed.

According to WOS, 78% the projects listed are compatible with Py3 but if you look at the numbers further you see that for the first 100 its 85% and then drops to 70% for the second 100. What do you think is going to happen when you look at the 3rd, 4th, etc 100 projects.

Now if you look at the list of projects a little more closely you will see that there is a lot of projects that I would consider as “infrastructure” projects. These projects are not the projects that your production software has dependencies on but are projects that help developers get their work done. They are all import projects but just not for your production code. From scanning the list I count 28 of them, 18 in the first 100 and 10 in the second. It is also of interest and comes as no surprise to me, that they all support Py3. After all developers tend to take care of their needs first.

Now if you remove these “infrastructure” projects from WOS you will find that 74% the projects listed are compatible with Py3, the first 100 its at 82% and then drops to 67% for the second 100.

Another thing to consider about the WOS website is that there are a number of influences that go into making the number of projects supporting Py3 look better. The PSF has funded a number of projects as well as a number of GSOC projects that have helped push the conversion of many of these projects. Also there have been a large number of individuals that have been pushing these projects to upgrade. Projects off this list however are not receiving all this extra attention.

I’m glad, that Concurrency is on the list (in GitHub), eventhough it’s the last one. We all know, concurrency is hard 😉 but why not implement an actor based concurrency model? I’m actually implementing this (using message passing using Queue) by myself whenever I need multi-threading in CPython. Why no provide that as a build feature allowing only immultible objects or copies to be passed?

I’m also disappointed that they are targeting the outdated form of the Python language. I made the switch to Python3 a while ago and I have been much happier with the results. I have to point out that multi-threading under Python 3.3 is much improved. I can hit all cores with IO bound threads. It’s not perfect but works well enough.

I’ve been waiting for PyPy to support Python 3.3 features so I can start experimenting with it. I was hoping that Pyston would be Python3 from the start. With them targeting Python2, it’s a non-starter.

Still, developing Pyston seems like a Herculean task, or it would have already been done. I’m presuming that before taking the decision of starting Pyston you looked at other statically compiled languages and discarded them. Could you please share which languages you did look at and decided to not use?

I think the Julia language is worth mentioning here, as it can be seen as one of the most (the most?) successfully example of how far a dynamic language can go in terms of performance.
Of course, it was designed from the very beginning with that objective in mind: http://julialang.org/blog/2012/02/why-we-created-julia/ . C-like performance seems something really hard to achieve in a language already so well established as Python (something not impossible to money tough (JavaScript)).

This is just a personal opinion. I think choose Python 2.7 is Right, because I think many module not Support Python 3, code migration is a boring and time-consuming process, Python 3 support can be completed at a later.

Not really. They said he didn’t write a single line so far.
Remember that when Unladen Swallow was announced (by Google engineers), he was working there and he quickly endorsed the project.
A few months later, it was abandoned.

That’s exactly what I feel about this project.
It’s a deja vu from Unladen Swallow. Looks like they haven’t done their homework and they ignore past failed attempts.
And I don’t understand why Guido is so quick to endorse these projects while, with pypy, he’s been always skeptical.

I totally agree with you mate. Whats more important is that Pypy is on its way to implement a GIL free python called pypy-stm. I feel like Dropbox is trying to flex its muscles or perhaps trying to be like FB?

Please note: Sometimes we blog about upcoming products or features before they’re released, but timing and exact functionality of these features may change from what’s shared here. The decision to purchase our services should be made based on features that are currently available.