Friday, June 9, 2017

PyPy v5.8 released

The PyPy team is proud to release both PyPy2.7 v5.8 (an interpreter supporting
Python 2.7 syntax), and a beta-quality PyPy3.5 v5.8 (an interpreter for Python
3.5 syntax). The two releases are both based on much the same codebase, thus
the dual release. Note that PyPy3.5 supports Linux 64bit only for now.

This new PyPy2.7 release includes the upstream stdlib version 2.7.13, and
PyPy3.5 includes the upstream stdlib version 3.5.3.

We fixed critical bugs in the shadowstack rootfinder garbage collector
strategy that crashed multithreaded programs and very rarely showed up
even in single threaded programs.

We added native PyPy support to profile frames in the vmprof statistical
profiler.

The struct module functions pack* and unpack* are now much faster,
especially on raw buffers and bytearrays. Microbenchmarks show a 2x to 10x
speedup. Thanks to Gambit Research for sponsoring this work.

Please let us know if your use case is slow, we have ideas how to make things
faster but need real-world examples (not micro-benchmarks) of problematic code.

Work sponsored by a Mozilla grant continues on PyPy3.5; numerous fixes from
CPython were ported to PyPy and PEP 489 was fully implemented. Of course the
bug fixes and performance enhancements mentioned above are part of both PyPy
2.7 and PyPy 3.5.

CFFI, which is part of the PyPy release, has been updated to an unreleased 1.10.1,
improving an already great package for interfacing with C.

Anyone using NumPy 1.13.0, must upgrade PyPy to this release since we implemented some previously missing C-API functionality. Many other c-extension modules now work with PyPy, let us know if yours does not.

As always, this release fixed many issues and bugs raised by the
growing community of PyPy users. We strongly recommend updating.

We would like to thank our donors and contributors, and
encourage new people to join the project. PyPy has many
layers and we need help with all of them: PyPy and RPython documentation
improvements, tweaking popular modules to run on PyPy, or general help
with making RPython’s JIT even better.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7 and CPython 3.5. It’s fast (PyPy and CPython 2.7.x performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
The PyPy 2.7 release supports:

What else is new?

There are many incremental improvements to RPython and PyPy, the complete listing is here.

Please update, and continue to help us make PyPy better.

Cheers, The PyPy team

The PyPy team is proud to release both PyPy2.7 v5.8 (an interpreter supporting
Python 2.7 syntax), and a beta-quality PyPy3.5 v5.8 (an interpreter for Python
3.5 syntax). The two releases are both based on much the same codebase, thus
the dual release. Note that PyPy3.5 supports Linux 64bit only for now.

This new PyPy2.7 release includes the upstream stdlib version 2.7.13, and
PyPy3.5 includes the upstream stdlib version 3.5.3.

We fixed critical bugs in the shadowstack rootfinder garbage collector
strategy that crashed multithreaded programs and very rarely showed up
even in single threaded programs.

We added native PyPy support to profile frames in the vmprof statistical
profiler.

The struct module functions pack* and unpack* are now much faster,
especially on raw buffers and bytearrays. Microbenchmarks show a 2x to 10x
speedup. Thanks to Gambit Research for sponsoring this work.

Please let us know if your use case is slow, we have ideas how to make things
faster but need real-world examples (not micro-benchmarks) of problematic code.

Work sponsored by a Mozilla grant continues on PyPy3.5; numerous fixes from
CPython were ported to PyPy and PEP 489 was fully implemented. Of course the
bug fixes and performance enhancements mentioned above are part of both PyPy
2.7 and PyPy 3.5.

CFFI, which is part of the PyPy release, has been updated to an unreleased 1.10.1,
improving an already great package for interfacing with C.

Anyone using NumPy 1.13.0, must upgrade PyPy to this release since we implemented some previously missing C-API functionality. Many other c-extension modules now work with PyPy, let us know if yours does not.

As always, this release fixed many issues and bugs raised by the
growing community of PyPy users. We strongly recommend updating.

We would like to thank our donors and contributors, and
encourage new people to join the project. PyPy has many
layers and we need help with all of them: PyPy and RPython documentation
improvements, tweaking popular modules to run on PyPy, or general help
with making RPython’s JIT even better.

What is PyPy?

PyPy is a very compliant Python interpreter, almost a drop-in replacement for CPython 2.7 and CPython 3.5. It’s fast (PyPy and CPython 2.7.x performance comparison) due to its integrated tracing JIT compiler.
We also welcome developers of other dynamic languages to see what RPython can do for them.
The PyPy 2.7 release supports:

5 comments:

Can we get a comprehensive update on Numpypy? It has gone really quiet since the days when Alex Gaynor used to talk at Pycon etc about the work which has been going on since what 2010/11? The repo has issues that are not looked at. I would really like an honest appraisal of what was learned in the Numpypy project and what is the future of Numpy (Scipy too) & PyPy because the situation for developers like myself is that we're caught between a rock and a hard place. PyPy consistently allows us to write code and explore algorithms in Python!! Whereas CPython forces you into C/Cython continually. PyPy is a great dream in my heart. What you guys are doing - allowing me to write Python and it be fast. What other language forces you so much to write in another language when performance is a consideration? The speed difference between Node.js and Python 3 is laughable. PyPy for the win!!!!

But....and it's a big but I am one of those devs who extensively is addicted to numeric arrays, not because I'm a 'quant' or an astronomer or rocket scientist but because Numpy arrays are simply better for many solutions than Python's other data structures. And once leveraged, giving that up to go to PyPy is impossible. It forces you to choose between numpy + slower python (CPython) or slower Numpy and faster python (PyPy).

Numpypy was a great dream, the best of both. But it seems to have failed, proven to be too difficult or does it simply need more money? I would appreciate a public update (if one exists, please link to it). Because the sadness for me is that a genuinely fast Python runtime will never be usable until the Numpy/Scipy world works and you get the fast python and as fast numpy.

I would really like to help, raise money whatever but maybe I'm out of the loop and the plan has changed?

Hi Albert. We have decided that a better route is to use upstream NumPy for compatibility. We are a small group, and reimplementing all of the c code in NumPy for Numpypy would be a never ending, close to impossible task.

However, we do have a different long-term plan to combine numpy and python. Since our c-api emulation layer is slow, perhaps we can "hijack" the most common python calls that cross that emulation border and make them fast. This would utilize much of NumPyPy but would mean that only a subset of the extensive NumPy library would need to be implemented and maintained. We have a branch that demonstrates a proof-of-concept for simple item access (ctypedef struct). Help on the PyPy project is always welcome, come to #pypy on IRC and we can discuss it further.