Nuitka Release 0.5.11

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler. Please see the page "What is Nuitka?" for an overview.

The last release represented a significant change and introduced a few
regressions, which got addressed with hot fix releases. But it also had a focus
on cleaning up open optimization issues that were postponed in the last release.

New Features

The filenames of source files as found in the __file__ attribute are
now made relative for all modes, not just standalone mode.

This makes it possible to put data files along side compiled modules in a
deployment. This solves Issue#170.

Bug Fixes

Local functions that reference themselves were not released. They now are.

Recent changes to code generation attached closure variable values to the
function object, so now they can be properly visited. This corrects
Issue#45. Fixed in 0.5.10.1
already.

Python2.6: The complex constants with real or imaginary parts -0.0 were
collapsed with constants of value 0.0. This became more evident after
we started to optimize the complex built-in. Fixed in 0.5.10.1 already.

complex(0.0,0.0)complex(-0.0,-0.0)# Could be confused with the above.

Complex call helpers could leak references to their arguments. This was a
regression. Fixed in 0.5.10.1 already.

Parameter variables offered as closure variables were not properly released,
only the cell object was, but not the value. This was a regression. Fixed in
0.5.10.1 already.

Compatibility: The exception type given when accessing local variable values
not initialized in a closure taking function, needs to be NameError and
UnboundLocalError for accesses in the providing function. Fixed in
0.5.10.1 already.

Fix support for "venv" on systems, where the system Python uses symbolic
links too. This is the case on at least on Mageia Linux. Fixed in
0.5.10.2 already.

Python3.4: On systems where long and Py_ssize_t are different (e.g.
Win64) iterators could be corrupted if used by uncompiled Python code. Fixed
in 0.5.10.2 already.

Compatiblity: The __closure__ attributes of functions was so far not
supported, and rarely missing. Recent changes made it easy to expose, so now
it was added. This corrects Issue#45.

MacOS: A linker warning about deprecated linker option -s was solved by
removing the option.

Compatibility: Nuitka was enforcing that the __doc__ attribute to be a
string object, and gave a misleading error message. This check must not be
done though, __doc__ can be any type in Python. This corrects Issue#177.

New Optimization

Variables that need not be shared, because the uses in closure taking
functions were eliminated, no longer use cell objects.

The try/except and try/finally statements now both have
actual merging for SSA, allowing for better optimization of code behind it.

deff():try:a=something()except:return2# Since the above exception handling cannot continue the code flow,# we do not have to invalidate the trace of "a", and e.g. do not have# to generate code to check if it's assigned.returna

Since try/finally is used in almost all re-formulations of complex
Python constructs this is improving SSA application widely. The uses of
try/except in user code will no longer degrade optimization and
code generation efficiency as much as they did.

The try/except statement now reduces the scope of tried block if
possible. When no statement raised, already the handling was removed, but
leading and trailing statements that cannot raise, were not considered.

deff():try:b=1a=something()c=1except:return2

This is now optimized to.

deff():b=1try:a=something()except:return2c=1

The impact may on execution speed may be marginal, but it is definitely
going to improve the branch merging to be added later. Note that c can
only be optimized, because the exception handler is aborting, otherwise it
would change behaviour.

The creation of code objects for standalone mode and now all code objects was
creating a distinct filename object for every function in a module, despite
them being same content. This was wasteful for module loading. Now it's done
only once.

Also, when having multiple modules, the code to build the run time filename
used for code objects, was calling import logic, and doing lookups to find
os.path.join again and again. These are now cached, speeding up the use
of many modules as well.

Cleanups

Nuitka used to have "variable usage profiles" and still used them to decide
if a global variable is written to, in which case, it stays away from doing
optimization of it to built-in lookups, and later calls.

The have been replaced by "global variable traces", which collect the traces
to a variable across all modules and functions. While this is now only a
replacement, and getting rid of old code, and basing on SSA, later it will
also allow to become more correct and more optimized.

The standalone now queries its hidden dependencies from a plugin framework,
which will become an interface to Nuitka internals in the future.

Testing

The use of deep hashing of constants allows us to check if constants become
mutated during the run-time of a program. This allows to discover corruption
should we encounter it.

The tests of CPython are now also run with Python in debug mode, but only on
Linux, enhancing reference leak coverage.

The CPython test parts which had been disabled due to reference cycles
involving compiled functions, or usage of __closure__ attribute, were
reactivated.

Organizational

Since Google Code has shutdown, it has been removed from the Nuitka git
mirrors.

Summary

This release brings exciting new optimization with the focus on the try
constructs, now being done more optimal. It is also a maintenance release,
bringing out compatibility improvements, and important bug fixes, and important
usability features for the deployment of modules and packages, that further
expand the use cases of Nuitka.

The git flow had to be applied this time to get out fixes for regression bug
fixes, that the big change of the last release brought, so this is also to
consolidate these and the other corrections into a full release before making
more invasive changes.

The cleanups are leading the way to expanded SSA applied to global variable
and shared variable values as well. Already the built-in detect is now based
on global SSA information, which was an important step ahead.