Friday, June 8, 2012

PyPy 1.9 - Yard Wolf

We're pleased to announce the 1.9 release of PyPy. This release brings mostly
bugfixes, performance improvements, other small improvements and overall
progress on the numpypy effort.
It also brings an improved situation on Windows and OS X.

Thank you all for proving that it is indeed possible for a small team of
programmers to get funded like that, at least for some
time. We want to include this thank you in the present release
announcement even though most of the work is not finished yet. More
precisely, neither Py3k nor STM are ready to make it in an official release
yet: people interested in them need to grab and (attempt to) translate
PyPy from the corresponding branches (respectively py3k andstm-thread).

Highlights

This release still implements Python 2.7.2.

Many bugs were corrected for Windows 32 bit. This includes new
functionality to test the validity of file descriptors; and
correct handling of the calling convensions for ctypes. (Still not
much progress on Win64.) A lot of work on this has been done by Matti Picus
and Amaury Forgeot d'Arc.

Improvements in cpyext, our emulator for CPython C extension modules.
For example PyOpenSSL should now work. We thank various people for help.

Sets now have strategies just like dictionaries. This means for example
that a set containing only ints will be more compact (and faster).

A lot of progress on various aspects of numpypy. See the numpy-status
page for the automatic report.

It is now possible to create and manipulate C-like structures using the
PyPy-only _ffi module. The advantage over using e.g. ctypes is that_ffi is very JIT-friendly, and getting/setting of fields is translated
to few assembler instructions by the JIT. However, this is mostly intended
as a low-level backend to be used by more user-friendly FFI packages, and
the API might change in the future. Use it at your own risk.

The non-x86 backends for the JIT are progressing but are still not
merged (ARMv7 and PPC64).

Handling of keyword arguments has been drastically improved in the best-case
scenario: proxy functions which simply forwards *args and **kwargs
to another function now performs much better with the JIT.

List comprehension has been improved.

JitViewer

There will be a corresponding 1.9 release of JitViewer which is guaranteed to work
with PyPy 1.9. See the JitViewer docs for details.

Cheers,
The PyPy Team

We're pleased to announce the 1.9 release of PyPy. This release brings mostly
bugfixes, performance improvements, other small improvements and overall
progress on the numpypy effort.
It also brings an improved situation on Windows and OS X.

Thank you all for proving that it is indeed possible for a small team of
programmers to get funded like that, at least for some
time. We want to include this thank you in the present release
announcement even though most of the work is not finished yet. More
precisely, neither Py3k nor STM are ready to make it in an official release
yet: people interested in them need to grab and (attempt to) translate
PyPy from the corresponding branches (respectively py3k andstm-thread).

Highlights

This release still implements Python 2.7.2.

Many bugs were corrected for Windows 32 bit. This includes new
functionality to test the validity of file descriptors; and
correct handling of the calling convensions for ctypes. (Still not
much progress on Win64.) A lot of work on this has been done by Matti Picus
and Amaury Forgeot d'Arc.

Improvements in cpyext, our emulator for CPython C extension modules.
For example PyOpenSSL should now work. We thank various people for help.

Sets now have strategies just like dictionaries. This means for example
that a set containing only ints will be more compact (and faster).

A lot of progress on various aspects of numpypy. See the numpy-status
page for the automatic report.

It is now possible to create and manipulate C-like structures using the
PyPy-only _ffi module. The advantage over using e.g. ctypes is that_ffi is very JIT-friendly, and getting/setting of fields is translated
to few assembler instructions by the JIT. However, this is mostly intended
as a low-level backend to be used by more user-friendly FFI packages, and
the API might change in the future. Use it at your own risk.

The non-x86 backends for the JIT are progressing but are still not
merged (ARMv7 and PPC64).

Handling of keyword arguments has been drastically improved in the best-case
scenario: proxy functions which simply forwards *args and **kwargs
to another function now performs much better with the JIT.

List comprehension has been improved.

JitViewer

There will be a corresponding 1.9 release of JitViewer which is guaranteed to work
with PyPy 1.9. See the JitViewer docs for details.

4 comments:

I have took a look at the mentioned numpypy table (http://buildbot.pypy.org/numpy-status/latest.html), and it lies in many ways. At first, some methods marked as "done" and undone yet, e.g. consider searchsorted: >>>> from numpypy import searchsorted>>>> searchsorted([1,2,3],[2,3])Traceback (most recent call last): File "", line 1, in File "/home/dmitrey/Install/pypy-c-jit-55492-ac392fb76904-linux/lib_pypy/numpypy/core/fromnumeric.py", line 763, in searchsorted raise NotImplementedError('Waiting on interp level method')NotImplementedError: Waiting on interp level method

(and AFAIK there are many other similar numpypy funcs that are present in dir(numpypy), but only raise NotImplementedError).

At 2nd, some funcs like all and any, also mentioned there as "done", don't work with "axis" parameter and thus also should be unmarked.

I have implemented them in AppLevel (thus PyPy developers refuce to commit them, but some users could be interested right now), see http://openopt.org/PyPy for more details and my sincere opinion on the situation.

The page is automatically generated - we should probably just disable those functions, I can't remember the exact reason why they're there in the first place.

When it comes to missing arguments - you just can't help it. It's an automatically generated page that should give only an overview.

As far as your patches go - yes, we need tests and we also need tests that cover corner cases. This is very important for us, we can live without the rest (like implementations on the interp-level). We do care about quality a lot.

hi fijal,as far as I remember, main reasons of PyPy developers (I don't remember namely) to reject my funcs propositions were AppLevel vs InterpLevel, not corner testcases (they even said "don't start the func, it must be InterpLevel"). Thus to speedup OpenOpt port on PyPy I went other way and as you probably have seen that some OpenOpt Suite functionality is already available in PyPy and works some times faster.

If apperplevel is ok for some of those funcs mentioned above, you or any other PyPy programmer can take anything from the code; as for me, I have lots of other things to do with my projects, especially now, before regular release, and thus cannot allocate time to create testcases for the numpy funcs.

BTW, what about fancy indexing with int arrays (https://bugs.pypy.org/issue1130) - when it will be implemented? It's very important for many Python projects and hangs for a long time already.