Friday, September 23, 2016

From September 5th to the 9th a group of Python core developers gathered for a sprint hosted at Instagram and sponsored by Instagram, Microsoft, and the Python Software Foundation. The goal was to spend a week working towards the Python 3.6b1 release, just in time for the Python 3.6 feature freeze on Monday, September 12, 2016. The inspiration for this sprint was the Need for Speed sprint held in Iceland a decade ago, where many performance improvements were made to Python 2.5. How time flies!

By any measurement, the sprint was extremely successful. All participants left feeling accomplished both in the work they did and in the discussions they held. Being in the same room encouraged many discussions related to various PEPs, and many design decisions were made. There was also the camaraderie of working in the same room together — typically most of us only see each other at the annual PyCon US, where there are other distractions that prevent getting to simply spend time together. (This includes the Python development sprints at PyCon, where the focus is more on helping newcomers contribute to Python — that’s why this sprint was not public.)

From a quantitative perspective, the sprint was the most productive week for Python ever! According to the graphs from the GitHub mirror of CPython, the week of September 4th saw more commits than the preceding 7 weeks combined! And in terms of issues, the number of open issues dropped by 62 with a total of 166 issues closed.

A large portion of the work performed during the sprint week revolved around various PEPs that had either not yet been accepted or were not yet fully implemented. In the end, 12 PEPs were either implemented from scratch or had their work completed during the week, out of Python 3.6’s total of 16 PEPs:

Some large projects were also worked on that are not represented as PEPs. For instance, Python 3.6 now contains support for DTrace and SystemTap. This will give people more tools to introspect and monitor Python. See the HOWTO for usage instructions and examples showing some of the new possibilities.

CPython also gained a more memory efficient dictionary implementation at the sprint. The new implementation shrinks memory usage of dictionaries by about 25% and also preserves insertion order, without speed penalties. Based on a proposal by Raymond Hettinger, the patch was written by INADA Naoki prior to the sprint but it was reviewed and heavily discussed at the sprint, as changing the underlying implementation of dictionaries can have profound implications on Python itself. In the end, the patch was accepted, directly allowing for PEP 468 to be accepted and simplifying PEP 520.

Work was also done on the Gilectomy (see the presentation on the topic from PyCon US for more background info on the project). Progress was made such that Python would run without any reference counting turned on (i.e. Python turned into a huge memory leak). Work then was started on trying the latest design on how to turn reference counting back on in a way that would allow Python to scale with the number of threads and CPU cores used. There’s still a long road ahead before the Gilectomy will be ready to merge though, and we even jokingly considered branding the result as Python 4.

Much of the work done during the sprint led not only to improvements in the language and library, but to better performance as well. A quick performance comparison between Python 3.5.2+ and 3.6b1+ under OS X shows that 3.6 is generally faster, with double-digit speed improvements not uncommon. Similar benchmarking under Windows 10 has been reported to show similar performance gains.

A huge thanks goes out to the participants of the sprints! They are listed below in alphabetical order, along with thanks to the organizations that helped finance their attendance. Many of them traveled to attend and gave up the US Labor Day holiday with their families to participate. In the end, we had participants from 3 countries on 2 continents (We actually invited more people from other countries and continents, but not everybody invited could attend.)

Brett Cannon (Microsoft)

Ned Deily

Steve Dower (Microsoft)

Larry Hastings

Raymond Hettinger

Senthil Kumaran

Łukasz Langa (Instagram)

R. David Murray

Benjamin Peterson

Davin Potts

Lisa Roach (Cisco)

Guido van Rossum

Yury Selivanov (magic.io)

Gregory P. Smith (Google)

Eric Snow (Canonical)

Victor Stinner (Red Hat)

Zachary Ware

Special thanks to Łukasz for making the event happen and to Larry for designing the logo.

Beta releases are intended to give the wider community the opportunityto test new features and bug fixes and to prepare their projects tosupport the new feature release. We strongly encourage maintainers ofthird-party Python projects to test with 3.6 during the beta phase andreport issues found to bugs.python.org as soon as possible. While therelease is feature complete entering the beta phase, it is possible thatfeatures may be modified or, in rare cases, deleted up until the startof the release candidate phase (2016-12-05). Our goal is have no changesafter rc1. To achieve that, it will be extremely important to get asmuch exposure for 3.6 as possible during the beta phase. Please keep inmind that this is a preview release and its use is not recommended forproduction environments

The next planned release of Python 3.6 will be 3.6.0b2, currentlyscheduled for 2016-10-03. More information about the release schedulecan be found here: