Search Results: "Matthias Klose"

4 June 2020

Welcome to the May 2020 report from the Reproducible Builds project.
One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. Nonetheless, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into seemingly secure software during the various compilation and distribution processes.
In these reports we outline the most important things that we and the rest of the community have been up to over the past month.

Recent years saw a number of supply chain attacks that leverage the increasing use of open source during software development, which is facilitated by dependency managers that automatically resolve, download and install hundreds of open source packages throughout the software life cycle.

This means that anyone can recreate the same binaries produced from our official release process. Now anyone can verify that the release binaries were created using the source code we say they were created from. No single person or computer needs to be trusted when producing the binaries now, which greatly reduces the attack surface for Sia users.

Synchronicity is a distributed build system for Rust build artifacts which have been published to crates.io. The goal of Synchronicity is to provide a distributed binary transparency system which is independent of any central operator.
The Comparison of Linux distributions article on Wikipedia now features a Reproducible Builds column indicating whether distributions approach and progress towards achieving reproducible builds.

Drop the (default) shell=False keyword argument to subprocess.Popen so that the potentially-unsafe shell=True is more obvious. []

Perform string normalisation in Black [] and include the Black output in the assertion failure too [].

Allow a bare try/except block when cleaning up temporary files with respect to the flake8 quality assurance tool. []

Rename in_dsc_path to dsc_in_same_dir to clarify the use of this variable. []

Abstract out the duplicated parts of the debian_fallback class [] and add descriptions for the file types. []

Various commenting and internal documentation improvements. [][]

Rename the Openssl command class to OpenSSLPKCS7 to accommodate other command names with this prefix. []

Misc:

Rename the --debugger command-line argument to --pdb. []

Normalise filesystem stat(2) birth times (ie. st_birthtime) in the same way we do with the stat(1) command s Access: and Change: times to fix a nondeterministic build failure in GNU Guix. (#74)

Ignore case when ordering our file format descriptions. []

Drop, add and tidy various module imports. [][][][]

In addition:

Jean-Romain Garnier fixed a general issue where, for example, LibarchiveMember s has_same_content method was called regardless of the underlying type of file. []

Daniel Fullmer fixed an issue where some filesystems could only be mounted read-only. (!49)

Emanuel Bronshtein provided a patch to prevent a build of the Docker image containing parts of the build s. (#123)

Mattia Rizzolo added an entry to debian/py3dist-overrides to ensure the rpm-python module is used in package dependencies (#89) and moved to using the new execute_after_* and execute_before_* Debhelper rules [].

Add a separate, canonical page for every new release. [][][]

Generate a latest release section and display that with the corresponding date on the homepage. []

Use Jekyll s absolute_url and relative_url where possible [][] and move a number of configuration variables to _config.yml [][].

Upstream patches
The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

Other tools
Elsewhere in our tooling:
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. In May, Chris Lamb uploaded version 1.8.1-1 to Debian unstable and Bernhard M. Wiedemann fixed an off-by-one error when parsing PNG image modification times. (#16)
In disorderfs, our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues, Chris Lamb replaced the term dirents in place of directory entries in human-readable output/log messages [] and used the astyle source code formatter with the default settings to the main disorderfs.cpp source file [].
Holger Levsen bumped the debhelper-compat level to 13 in disorderfs [] and reprotest [], and for the GNU Guix distribution Vagrant Cascadian updated the versions of disorderfs to version 0.5.10 [] and diffoscope to version 145 [].

Juri Dispan:

Testing framework
We operate a large and many-featured Jenkins-based testing framework that powers tests.reproducible-builds.org that, amongst many other tasks, tracks the status of our reproducibility efforts as well as identifies any regressions that have been introduced. Holger Levsen made the following changes:

System health status:

Improve page description. []

Add more weight to proxy failures. []

More verbose debug/failure messages. [][][]

Work around strangeness in the Bash shell let VARIABLE=0 exits with an error. []

Fail loudly if there are more than three .buildinfo files with the same name. []

Document how to reboot all nodes in parallel, working around molly-guard. []

Further work on a Debian package rebuilder:

Workaround and document various issues in the debrebuild script. [][][][]

Improve output in the case of errors. [][][][]

Improve documentation and future goals [][][][], in particular documentiing two real world tests case for an impossible to recreate build environment [].

Find the right source package to rebuild. []

Increase the frequency we run the script. [][][][]

Improve downloading and selection of the sources to build. [][][]

Improve version string handling.. []

Handle build failures better. []. []. []

Also consider architecture all .buildinfo files. [][]

In addition:

kpcyrd, for Alpine Linux, updated the alpine_schroot.sh script now that a patch for abuild had been released upstream. []

Alexander Couzens of the OpenWrt project renamed the brcm47xx target to bcm47xx. []

Mattia Rizzolo fixed the printing of the build environment during the second build [][][] and made a number of improvements to the script that deploys Jenkins across our infrastructure [][][].

Lastly, Vagrant Cascadian clarified in the documentation that you need to be user jenkins to run the blacklist command [] and the usual build node maintenance was performed was performed by Holger Levsen [][][], Mattia Rizzolo [][] and Vagrant Cascadian [][][].

To make the results accessible, storable and create tools around them, they should all follow the same schema, a reproducible builds verification format. The format tries to be as generic as possible to cover all open source projects offering precompiled source code. It stores the rebuilder results of what is reproducible and what not.

Do you own your Bitcoins or do you trust that your app allows you to use your coins while they are actually controlled by them ? Do you have a backup? Do they have a copy they didn t tell you about? Did anybody check the wallet for deliberate backdoors or vulnerabilities? Could anybody check the wallet for those?

Elsewhere, Leo had posted instructions on his attempts to reproduce the binaries for the BlueWallet Bitcoin wallet for iOS and Android platforms.
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:

IRC: #reproducible-builds on irc.oftc.net.

This month s report was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Jelle van der Waa and Vagrant Cascadian. It was subsequently reviewed by a bunch of Reproducible Builds folks on IRC and the mailing list.

3 November 2017

Today I spent some time in the morning seeing how one would install the JDK
on Linux distributions. This is to create a little comparative tutorial to
teach introductory Java.
Installing the JDK is, thanks to the OpenJDK developers in Debian and Ubuntu
(Matthias Klose and helpers), a very easy task. You simply type something
like:

apt-get install openjdk-8-jdk

Since for a student it is better to have everything for experiments, I
install the full version, not only the
-headless version. Given my familiarity with
Debian/Ubuntu, I didn't have to think about the way of installing it, of
course.
But as this is a tutorial meant to be as general as I can, I tried also to
include instructions on how to install Java on other distributions. The
first two that came to my mind were openSUSE and Fedora.
Both use the RPM package format for their "native" packages (in the same
sense that Debian uses DEB packages for "native" packages). But they use
different higher-level tools to install such packages: Fedora uses a tool
called dnf, while openSUSE uses zypper.
To try these distributions, I got their netinstall ISOs and used
qemu/kvm to install on a virtual machine. I used the following to
install/run the virtual machines (the example below, is, of course, for openSUSE):

The names of the packages also change from one distribution to another. On
Fedora, I had to use:

dnf install java-1.8.0-openjdk-devel

On openSUSE, I had to use:

zypper install java-1_8_0-openjdk-devel

Note that one distribution uses dots in the names of the packages while the
other uses underscores.
One interesting thing that I noticed with dnf was that, when I used it, it
automatically refreshed the package lists from the network, something which
I forgot, and it was a pleasant surprise. I don't know about zypper, but I
guess that it probably had fresh indices when the installation finished.
Both installations were effortless after I knew the names of the packages to
install.
Oh, BTW, in my 5 minute exploration with these distributions, I noticed that
if you don't want the JDK, but only the JRE, then you omit the -devel
suffix. It makes sense when you think about it, for consistency with other
packages, but Debian's conventions also make sense (JRE with -jre suffix,
JDK with -jdk suffix).
I failed miserably to use Fedora's prebaked, vanilla
cloud image, as I couldn't login on this image and I decided
to just install the whole OS on a fresh virtual machine.
I don't have instructions on how to install on Gentoo nor on
Arch, though.
I now see how hard it is to cover instructions/provide software for as many
distributions as you wish, given the multitude of package managers,
conventions etc.

On Saturday 28th October, Chris Lamb will present at freenode.live in Bristol, UK.

From October 31st November 2nd we will be holding the
3rd Reproducible Builds summit
in Berlin, Germany. If you are working in the field of reproducible builds, you should definitely
attend. Please see our public invitation mail and contact us if you have any questions.

New York University sessions
A three week session will be held at New York University to work on
reproducibilty issues in conjunction with the reproducible builds community.
Students from the Application Security course will be working for two weeks to work on the reproducible builds effort.

On Tuesday 24th Oct Ed Maste from FreeBSD will be presenting some reproducible
builds work for students.

On From Tuesday 24th of October to Monday 7th of November students will work
on fixing reproducibility issues brought up by the community. A milestone
presentation will be held by Santiago Torres-Arias and Preston Moore.

On Tuesday 7th November Holger Levsen will join the NYU team to wrap up the work.

Reviews of unreproducible packages
41 package reviews have been added, 119 have been updated and 54 have been removed in this week,
adding to our knowledge about identified issues. 2 issue types were removed as they were fixed:

3 October 2017

Not so long ago I went to effectively recompile NetworkManager and fix up minor bug in it. It built fine across all architectures, was considered to be installable etc. And I was expecting it to just migrate across. At the time, glibc was at 2.26 in artful-proposed and NetworkManager was built against it. However release pocket was at glibc 2.24. In Ubuntu we have a ProposedMigration process in place which ensures that newly built packages do not regress in the number of architectures built for; installable on; and do not regress themselves or any reverse dependencies at runtime.

Thus before my build of NetworkManager was considered for migration, it was tested in the release pocket against packages in the release pocket. Specifically, since package metadata only requires glibc 2.17 NetworkManager was tested against glibc currently in the release pocket, which should just work fine....

At first I only saw failing tests, which I thought is transient failure. Thus they were retried a few times. Then I looked at the autopkgtest log and saw above error messages. Perplexed, I have started a lxd container with ubuntu artful, enabled proposed and installed just network-manager from artful-proposed and indeed a simple NetworkManager --help failed with above error from linker.

I am too young to know what dependency-hell means, since ever since I used Linux (Ubuntu 7.04) all glibc symbols were versioned, and dpkg-shlibdeps would generate correct minimum dependencies for a package. Alas in this case readelf confirmed that indeed /usr/sbin/NetworkManager requires 2.25 and dpkg depends is >= 2.17.

Further reading readelf output I checked that all of the glibc symbols used are 2.17 or lower, and only the "Version needs section '.gnu.version_r'" referenced GLIBC_2.25 symbol. Inspecting dpkg-shlibdeps code I noticed that it does not parse that section and only searches through the dynamic symbols used to establish the minimum required version.

Things started to smell fishy. On one hand, I trust dpkg-shlibdeps to generate the right dependencies. On the other hand I also trust linker to not tell lies either. Hence I opened a Debian BTS bug report about this issue.

At this point, I really wanted to figure out where the reference to 2.25 comes from. Clearly it was not from any private symbols as then the reference would be on 2.26. Checking glibc abi lists I found there were only a handful of symbols marked as 2.25

First of all it seems like network-manager includes a partial embedded copy of systemd. Secondly that code is compiled into a temporary library and has autconf detection logic to use explicit_bzero. It also has an embedded implementation of explicit_bzero when it is not available in libc, however it does not have FORTIFY_SOURCES implementation of said function (__explicit_bzero_chk) as was later pointed out to me. And whilst this function is compiled into an intermediary noinst library, no functions that use explicit_bzero are used in the end by NetworkManger binary. To proof this, I've dropped all code that uses explicit_bzero, rebuild the package against glibc 2.26, and voila it only had Version reference on glibc 2.17 as expected from the end-result usage of shared symbols.

At this point toolchain bug was a suspect. It seems like whilst explicit_bzero shared symbol got optimised out, the version reference on 2.25 persisted to the linked binaries. At this point in the archive a snapshot version of binutils was in use. And in fact forcefully downgrading bintuils resulted in correct compilation / versions table referencing only glibc 2.17.

Mathias then took over a tarball of object files and filed upstream bug report against bintuils: "[2.29 Regression] ld.bfd keeps a version reference in .gnu.version_r for symbols which are optimized out". The discussion in that bug report is a bit beyond me as to me binutils is black magic. All I understood there was "we moved sweep and pass to another place due to some bugs", doing that introduced this bug, thus do multiple sweep and passes to make sure we fix old bugs and don't regress this either. Or something like that. Comments / Better description of the bintuils fix are welcomed.

Binutils got fixed by upstream developers, cherry-picked into debian, and ubuntu, network-manager got rebuild and everything is wonderful now. However, it does look like unused / deadend code paths tripped up optimisations in the toolchain which managed to slip by distribution package dependency generation and needless require a higher up version of glibc. I guess the lesson here is do not embed/compile unused code. Also I'm not sure why network-manager uses networkd internals like this, and maybe systemd should expose more APIs or serialise more state into /run, as most other things query things over dbus, private socket, or by establishing watches on /run/systemd/netif. I'll look into that another day.

Thanks a lot to Guillem Jover, Matthias Klose, Alan Modra, H.J. Lu, and others for getting involved. I would not be able to raise, debug, or fix this issue all by myself.

2 November 2016

What happened in the Reproducible
Builds effort between Sunday
October 23 and Saturday October 29 2016:
Upcoming events
The second Reproducible Builds World Summit will be held from December
13th-15th in Berlin! See
the link for more details.
Other events:
Introduction to Reproducible
Builds
- Vagrant Cascadian will be presenting at the SeaGL.org Conference In
Seattle, USA on November 12th, 2016.
Reproducible Debian Hackathon - A small hackathon organized in Boston, USA on
December 3rd and 4th. If you are interested in attending, contact Valerie Young
- spectranaut in the #debian-reproducible IRC channel on irc.oftc.net.
IRC meeting
The next IRC meeting will be held on 2016-11-01 at 18:00 UTC. The meeting after
that will be held on 2016-11-15 at 18:00 UTC.
Reproducible work in other projects
Ximin Luo has had his fix to bug
77985accepted into
GCC. This is needed to be able to write test cases for patches to make GCC
produce debugging symbols that are reproducible regardless of the build path.
There was continued
discussion
on the mailing list regarding our build path proposals. It has now been decided
to use an environment variable SOURCE_PREFIX_MAP instead of the older
proposal SOURCE_ROOT_DIR. This would be similar to GCC's existing
-fdebug-prefix-map option, which allows for better disambiguation between
paths from different packages.
mandoc's makewhatis is now
reproducible.
It is used by all the BSDs, including
FreeBSD, as well as Alpine
Linux and Void Linux.
Packages reviewed and fixed, and bugs filed
Chris Lamb:

Reviews of unreproducible packages
145 package reviews have been added, 608 have been updated and 94 have been
removed in this week, adding to our knowledge about identified
issues.
3 issue types have been updated:

Weekly QA work
During of reproducibility testing, some FTBFS bugs have been detected and
reported by:

Chris Lamb (17)

Matthias Klose (2)

tests.reproducible-builds.org
Debian:

Valerie improved the SQL code so that the scheduler job again runs within
minutes. She did the same to the job updating the notes about known issues,
though this job still runs 12min and not 2min as it used to do

Thanks to a patch from Chris, which was improved by dkg and h01ger after
discussions on our list, the .buildinfo files submitted to
buildinfo.debian.net are now GPG signed by our build nodes.

General:

Holger fixed the NetBSD and coreboot jobs which were broken due to work on
the LEDE+OpenWRT jobs.

As squid on jessie/i386 (but not on jessie/amd64) crashes frequently, we have
have monitoring for this and Holger fixed a subtile bug there.

12 October 2016

Hello folks! My team spent hours and hours beating our head against a Sonar deployment problem on Ubuntu Trusty (14.04 LTS). I thought I might share our findings so that you won t have to!
As you probably know, Trusty only makes Java Development Kit 1.7 available on the stock installation. The current stable version of the Java is 1.8. The way we install this is to use the OpenJDK PPA, generously uploaded by our dear friend Matthias Klose.
To make things even more exciting, a modern Maven is not available on this platform. And so we use the stock Maven 3.3.9 tarball distribution. This tarball distribution does not integrate well with Debian, and so, when we tell the system using sudo update-java-alternatives -s /usr/lib/jvm/java-1.8.0-openjdk-amd64 that we wish to use Java 1.8 as our default system JDK, it does not get the message.
The only way to reliably let Maven know which java you wish to use is to set the JAVA_HOME environment variable. In order to do this within the Jenkins environment, one must select the JDK one wishes to use:
To make things worse, this option is not, as one might expect, available for editing in a stock Jenkins 2.x installation. In Jenkins 1.x, one would be able to specify which java one wished to use just by specifying openjdk8 in a field. With Jenkins 2.x, the field does not exist unless a configuration option in an unrelated form is set.
So! One should first select Manage Jenkins -> Global Tool Configuration:
Once this form is open, look for the JDK installations button:
Click it very thoroughly just once.
You ll be presented with a form into which you may enter details about the various JDKs your build executors may have access to. You ll refer to them in your job configuration by the value of their Name field, and when executing the build, Jenkins will set JAVA_HOME to the value of the (you guessed it) JAVA_HOME field:
Once these entries are made, they can be selected in two place.
1) on the ZMQ Event Publisher:
2) in the post-build actions under SonarQube analysis with Maven (advanced)
And that s how it s done!
Here s the details from my colleague, Thanh:
https://lists.fd.io/pipermail/honeycomb-dev/2016-October/000387.html

21 July 2016

What happened in the Reproducible
Builds effort between June 26th and July 2nd 2016:
Read on to find out why we're lagging some weeks behind !
GSoC and Outreachy updates

Ceridwen
described
using autopkgtest code to communicate with containers and how to test the container handling.

reprotest 0.1 has been accepted into Debian unstable, and any user
reports, bug reports, feature requests, etc. would be appreciated.
This is still an alpha release, and nothing is set in stone.

Toolchain fixes

Matthias Klose uploaded doxygen/1.8.11-3 to Debian unstable (closing
#792201) with the
upstream patch
improving SOURCE_DATE_EPOCH support by using UTC as timezone when parsing the
value. This was the last patch we were carrying in our repository, thus this
upload obsoletes the version in our experimental repository.

cmake/3.5.2-2 was uploaded by Felix Geyer, which sorts file lists obtained
with file(GLOB).

With the doxygen upload we are now down to only 2 modified packages in our
repository: dpkg and rdfind.
Weekly reports delay and the future of statistics
To catch up with our backlog of weekly reports we have decided to skip some of the
statistics for this week. We might publish them in a future report, or we might
switch to a format where we summarize them more (and which we can create (even) more
automatically), we'll see.
We are doing these weekly statistics because we believe it's appropriate and
useful to credit people's work and make it more visible. What do you think? We would
love to hear your thoughts on this matter! Do you read these statistics? Somewhat?
Actually, thanks to the power of notmuch, Holger came up with what you can
see below, so what's missing for this week are the uploads fixing irreprodubilities.
Which we really would like to show for the reasons stated above and because we really
really need these uploads to happen
But then we also like to confirm the bugs are really gone, which (atm) requires manual
checking, and to look for the words "reproducible" and "deterministic"
(and spelling variations) in debian/changelogs of all uploads, to spot reproducible work
not tracked via the BTS.
And we still need to catch up on the backlog of weekly reports.
Bugs submitted with reproducible usertags
It seems DebCamp in Cape Town was hugely successful and made some people get a lot of work done:
61 bugs have been filed with reproducible builds usertags and 60 of them had patches:

#829365 against pdl by Reiner Herrmann: please make the build reproducible.

#828216 against pyinfra by Daniel Stender: assertion error with timestamps.

Package reviews
437 new reviews have been added (though most of them were just linking the bug, "only" 56 new issues in packages were found), an unknown number has been been updated and 60 have been removed in this week, adding to our knowledge about identified issues.
4 new issue types have been found:

#829115 against diffoscope by Axel Beckert and Mattia Rizzolo:: /comparators/ps.py: TypeError: cannot use a string pattern on a bytes-like object.

strip-nondeterminism development

Chris Lamb made sure that .zhfst files are treated as ZIP files.

tests.reproducible-builds.org

Mattia Rizzolo uploaded pbuilder/0.225.1~bpo8+1 to jessie-backports
and it has been installed on all build nodes. As a consequence all armhf and
i386 builds will be done with eatmydata; this will hopefully cut down
the build time by a noticable factor.

Misc.
This week's edition was written by Mattia Rizzolo, Reiner Herrmann, Ceridwen and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

18 June 2016

Wheezy's LTS period started a few weeks ago and the LTS team had to make an
early support decision concerning the Java eco-system since Wheezy ships two Java
runtime environments OpenJDK 6 and
OpenJDK 7. (To be fair, there are
actually three but gcj has
been superseded by OpenJDK a long time ago and the latter should be preferred
whenever possible.)
OpenJDK 6 is currently maintained by Red Hat and we mostly rely on their
upstream work as well as on package updates from Debian's maintainer Matthias Klose and
Tiago St rmer Daitx from Ubuntu. We already knew that both intend
to support OpenJDK 6 until April 2017 when Ubuntu 12.04 will reach its
end-of-life. Thus we had basically two options, supporting OpenJDK 6 for another
twelve months or dropping support right from the start. One of my first steps
was to ask for feedback and advice on
debian-java since
supporting only one JDK seemed to be the more reasonable solution. We agreed on warning users
via various channels about the intended change, especially about possible
incompatibilities
with OpenJDK 7. Even Andrew Haley, OpenJDK 6 project lead, participated in the discussion and confirmed that, while still supported, OpenJDK 6 security
releases are "always the last in the queue when there is urgent work to be
done".
I informed debian-lts about
my findings and issued a call for tests later.
Eventually we decided to concentrate our efforts on OpenJDK
7 because we are confident that for the
majority of our users one Java implementation is sufficient during a stable
release cycle. An immediate positive effect in making OpenJDK 7 the default is that
resources can be relocated to more pressing issues. On the other hand we were
also forced to make compromises. The switch to a newer default implementation
usually triggers a major transition with dozens of FTBFS bugs and the OpenJDK 7
transition
was no exception. I pondered about the usefulness of fixing all these bugs for
Wheezy LTS again and focussing on runtime issues instead and finally decided
that the latter was both more reasonable and more economic.
Different from regular default Java changes, users will still be able to use OpenJDK 6 to
compile their packages and the security impact for development systems is in general neglectable.
More important was to avoid runtime installations of OpenJDK 6. I identified eighteen
packages that strictly depended on the now obsolete JRE and fixed those
issues
on 4 May 2016 together with an update of
java-common and announced the switch to
OpenJDK 7 with a Debian NEWS file.
If you are not a regular reader of Debian
news and also not subscribed to
debian-lts,
debian-lts-announce or
debian-java, remember 26 June
2016 is the day when OpenJDK 7 will be made the default Java implementation in
Wheezy LTS. Of course there is no need to wait. You can switch right now:

sudo update-alternatives --config java

15 June 2016

What happened in the Reproducible
Builds effort between June 5th and June 11th 2016:
Media coverage
Ed Maste gave a talk at BSDCan 2016 on
reproducible builds
(slides,
video).
GSoC and Outreachy updates
Weekly reports by our participants:

Scarlett Clark
worked on making some packages reproducible, focusing on KDE backend and
utility programs.

Ceridwen
published an initial design for the interface for reprotest, including a
discussion on different types of build variations and the difficulties of
specifying certain types of variations.

Valerie Young improved documentation for
building our tests website, began migrating Debian-specific pages into a new
namespace, and planned future work around its navigation.

Documentation update
- Ximin Luo proposed a modification
to our SOURCE_DATE_EPOCH spec explaining FORCE_SOURCE_DATE.
Some upstream build tools (e.g. TeX, see below) have expressed a desire to
control which cases of embedded timestamps should obey SOURCE_DATE_EPOCH.
They were not convinced by our arguments on why this is a bad idea, so we
agreed on an environment variable FORCE_SOURCE_DATE for them to implement
their desired behaviour - named generically, so that at least we can set it
centrally. For more details, see the text just linked. However, we strongly
urge most build tools not to use this, and instead obey SOURCE_DATE_EPOCHunconditionally in all cases.
Toolchain fixes

TeX Live 2016 released
with SOURCE_DATE_EPOCH support for all engines except LuaTeX and original TeX.

Valerie Young namespaced the Debian-specific pages to /debian/
namespace, with redirects to
for the previous URLs.

Holger Levsen improved the reliability of build jobs: the availability of
both build nodes (for a given build) is now being tested when a build job is
started, to better cope when one of the 25 build nodes go down for some reason.

Ximin Luo improved the index of identified
issues to
include the total popcon scores of each issue, which is now also used for
sorting that page.

Misc.
Last week we also learned about progress of reproducible builds in FreeBSD. Ed Maste announced a change to record the build timestamp during ports building, which is required for later reproduction.
This week's edition was written by Reiner Herrman, Holger Levsen and Chris Lamb and reviewed by a bunch of Reproducible builds folks on IRC.

Martin Michlmayr had a talk in which he presented an overview about innovations and changes in Debian in the last years. Martin expressed his disappointment that there was no talk from us in Vienna (we'll fix this at DebConf16 in Cape Town) and described the reproducible builds work as "a real innovation". His talk is very much worth seeing, whatever your current perspective, it might change your view on Debian.

Ben Hutchings explains how Secure Boot will use signed kernels via separate signature packages and how this was designed with reproducible builds in mind.

Misc.
Amongst the 29 interns who will work on Debian through GSoC and Outreachy there are four who will be contributing to Reproducible Builds for Debian and Free Software. We are very glad to welcome ceridwen, Satyam Zode, Scarlett Clark and Valerie Young and look forward to working together with them the coming months (and maybe beyond)!
This week's edition was written by Reiner Herrmann and Holger Levsen and reviewed by a bunch of Reproducible builds folks on IRC.

#808388 on buzztrax by Chris Lamb: implement support for SOURCE_DATE_EPOCH.

reproducible.debian.net
Packages in experimental are now tested on armhf. (h01ger)
Arch Linux packages in the multilib and community repositories (4,000 more source packages) are also being tested. All of these test results are better analyzed and nicely displayed together with each package. (h01ger)
For Fedora, build jobs can now run in parallel. Two are currently running, now testing reproducibility of 785 source packages from Fedora 23.
mock/1.2.3-1.1 has been uploaded to experimental to better build RPMs. (h01ger)
Work has started on having automatic build node pools to maximize use of armhf build nodes. (Vagrant Cascadian)
diffoscope development
Version 43 has been released on December 15th. It has been dubbed as epic! as it contains many contributions that were written around the summit in Athens.
Baptiste Daroussin found that running diffoscope on some Tar archives could overwrite arbitrary files. This has been fixed by using libarchive instead of Python internal Tar library and adding a sanity check for destination paths. In any cases, until proper sandboxing is implemented, don't run diffosope on unstrusted inputs outside an isolated, throw-away system.
Mike Hommey identified that the CBFS comparator would needlessly waste time scanning big files. It will now not consider any files bigger than 24 MiB 8 MiB more than the largest ROM created by coreboot at this time. An encoding issue related to Zip files has also been fixed. (Lunar)
New comparators have been added: Android dex files (Reiner Herrmann), filesystem images using libguestfs (Reiner Herrmann), icons and JPEG images using libcaca (Chris Lamb), and OS X binaries (Clemens Lang). The comparator for Free Pascal Compilation Unit will now only be used when the unit version matches the compiler one. (Levente Polyak)
A new multi-file HTML output with on-demand loading of long diffs is available through the --html-dir option. On-demand loading requires jQuery which path can be specified through the --jquery option. The diffs can also be simply browsed for non-JavaScript users or when jQuery is not available. (Joachim Breitner)
Portability toward other systems has been improved: old versions of GNU diff are now supported (Mike McQuaid), suggestion of the appropriate locale is now the more generic en_US.UTF-8 (Ed Maste), the --list-tools option can now support multiple systems (Mattia Rizzolo, Levente Polyak, Lunar).
Many internal changes and code clean-ups have been made, paving the way for parallel processing. (Lunar)
Version 44 was released on December 18th fixing an issue affecting .deb lacking a md5sums file introduced in a previous refactoring (Lunar). Support has been added for Mozilla optimized Zip files. (Mike Hommey). The HTML output has been optimized in size (Mike Hommey, Esa Peuha, Lunar), speed (Lunar), and will now properly number lines (Mike Hommey). A message will always be displayed when lines are ignored at the end of a diff (Lunar). For portability and consistency, Python os.walk() function is now used instead of find to perform directory listing. (Lunar)
Documentation update
Package reviews
143 reviews have been removed, 69 added and 22 updated in the previous week.
Chris Lamb reported 12 new FTBFS issues.
News issues identified this week: random_order_in_init_py_generated_by_python-genpy, timestamps_in_copyright_added_by_perl_dist_zilla, random_contents_in_dat_files_generated_by_chasen-dictutils_makemat, timestamps_in_documentation_generated_by_pandoc.
Chris West did some improvements on the scripts used to manage notes in the misc repository.
Misc.
Accounts of the reproducible builds summit in Athens were written by Thomas Klausner from NetBSD and Hans-Christoph Steiner from The Guardian Project.
Some openSUSE developers are working on a hackweek on reproducible builds which was discussed on the opensuse-packaging mailing-list.

11 December 2015

The first reproducible world
summit was held in
Athens, Greece, from December 1st-3rd with the support of the
Linux Foundation, the Open Tech Fund, and Google. Faidon Liambotis has been an
amazing help to sort out all local details. People at ImpactHub
Athens have been perfect hosts.
Nearly 40 participants from 14 different free software project had very busy
days sharing knowledge, building understanding, and producing actual patches.
Anyone interested in cross project discussions should join the rb-general mailing-list.
What follows focuses mostly on what happened for Debian this previous week. A
more detailed report about the summit will follow soon. You can also read the ones from
Joachim Breitner from
Debian,
Clemens Lang from
MacPorts,
Georg Koppen from
Tor,
Dhiru Kholia from Fedora,
and Ludovic Court s wrote one for Guix and for the GNU project.
Infrastructure
Several discussions at the meeting helped refine a shared understanding of what
kind of information should be recorded on a build, and how they could be used.
Daniel Kahn Gillmor sent a detailed update
on how .buildinfo
files
should become part of the Debian archive.
Some key changes compared to what we had in mind at DebConf15:

Two .buildinfo with different environment information can attest to the
same exact binary artifact.

Multiple .buildinfo files can coexist for the same .deb as long as the listed
checksums match the source and binary package in the archive.

.buildinfo can be signed in-line to certify where a build comes from.

Hopefully, ftpmasters will be able to comment on the updated proposal soon.
Packages fixed
The following packages have become reproducible due to changes in their
build dependencies:
fades,
triplane,
caml-crush,
globus-authz.
The following packages became reproducible after getting fixed:

#806974 on xpra by Reiner Herrmann: interpret the changelog date as UTC.

#807051 on why by Valentin Lorentz: removes extra timestamps from the build system.

akira sent proposals on how to make bash reproducible.
Alexander Couzens submitted a patch upstream to add support for SOURCE_DATE_EPOCH in grub image generator (#787795).
reproducible.debian.net
An issue with some armhf build nodes was tracked down to a bad interaction between uname26 personality and new glibc (Vagrant Cascadian).
A Debian package was created for koji, the RPM building and tracking system used by Fedora amongst others. It is currently waiting for review in the NEW queue. (Ximin Luo, Marek Marczykowski-G recki)
diffoscope development
diffoscope now has a dedicated mailing list to better accommodate its growing user and developer base.
Going through diffoscope's guts together enabled several new contributors. Baptiste Daroussin, Ed Maste, Clemens Lang, Mike McQuaid, Joachim Breitner all contributed their first patches to improve portability or add new features. Regular contributors Chris Lamb, Reiner Herrmann, and Levente Polyak also submitted improvements.
The next release should support more operating systems, filesystem image comparison via libguestfs, HTML reports with on-demand loading, and parallel processing for the most noticeable improvements.
Package reviews
27 reviews have been removed, 17 added and 14 updated in the previous week.
Chris Lamb and Val Lorentz filed 4 new FTBFS reports.
Misc.
Baptiste Daroussin has started to implement support for SOURCE_DATE_EPOCH in FreeBSD in libpkg and the ports tree.
Thanks Joachim Breitner and h01ger for the pictures.

The package is not in yet in unstable, but linux/4.2-1~exp1 is now reproducible! Kudos to Ben Hutchings, and most fixes are already merged upstream.
Some uploads fixed some reproducibility issues but not all of them:

#797871 on xbae by Chris Lamb: use date of the latest debian/changelog entry as build time.

reproducible.debian.net
Some bugs that prevented packages to build successfully in the remote builders have been fixed. (h01ger)
Two more amd64 build jobs have been removed from the Jenkins host in favor of six more on the new remote nodes. (h01ger)
The munin graphs currently looks fine, so more amd64 jobs will probably be added in the next week.
diffoscope development
Version 32 of diffoscope has been released on September 3rd with the following new features:

A new --fuzzy-threshold option to specify the TLSH score used as cut-off
for fuzzy matching. Specifying 0 will disable fuzzy-matching entirely.
Suggested by Jakub Wilk.

A new --new-file option to treat absent files as empty. This make diffoscope a great
tool to look at the content of an archive at once by comparing it with a non-existent
file (example).
Suggested by Jakub Wilk.

Comparisons of symlinks and devices given on the command line is now possible.

21 July 2015

I recently got access to several ProLiant m400 ARM64 servers at work.
Since Debian is currently working on the migration to GCC 5, I thought it
would be nice to rebuild the Debian archive on ARM64 to see if GCC 5 is
ready. Fortunately, I found no obvious compiler errors.
During the process, I noticed several areas where ARM64 support can be
improved. First, a lot of packages failed to build due to missing
dependencies. Some missing dependencies are libraries or tools that have
not been ported to ARM64 yet, but the majority was due to the lack of
popular programming languages on ARM64. This requires upstream porting
work, which I'm sure is going on already in many cases. Second, over 160
packages failed to build due to out-of-date autoconf and libtool scripts.
Most of these bugs have been reported over a year ago by the
ARM64 porters (Matthias Klose from Canonical/Ubuntu and Wookey from
ARM/Linaro) and the PowerPC porters, but unfortunately they haven't been
fixed yet.
Finally, I went through all packages that list specific architectures in
debian/control and filed wishlist bugs on those that looked
relevant to ARM64. This actually prompted some Debian and upstream
developers to implement ARM64 support, which is great!

19 July 2015

after the release is before the release. or: long time no RC bug
report.
after the jessie release I spent most of my Debian time on work
in the Debian Perl
Group. we tried to get down the list of new upstream releases (from over
500 to currently 379; unfortunately the CPAN never sleeps), we were &
still are busy preparing for the Perl
5.22 transition (e.g. we uploaded something between 300 & 400
packages to deal with Module::Build & CGI.pm being removed from perl
core; only team-maintained packages so far), & we had a pleasant &
productive sprint
in Barcelona in May. & I also tried to fix some of the RC bugs
in our packages which popped up over the previous months.
yesterday & today I finally found some time to help with the GCC 5 transition, mostly by making
QA or Non-Maintainer Uploads with patches that already were in the BTS.
a big thanks especially to the team at HP which provided a couple
dozens patches!
& here's the list of RC bugs I've worked on in the last 3 months:

4 May 2015

Debian Jessie has been released on April 25th, 2015. This has opened the
Stretch development cycle. Reactions to the idea of making Debian
build reproducibly have
been pretty enthusiastic. As the pace is now likely to be even faster,
let's see if we can keep everyone up-to-date on the developments.
Before the release of Jessie
The story goes back a long
way
but a formal announcement to the
project
has only been sent in February 2015.
Since then, too much work has happened to make a complete report, but
to give some highlights:

New variations are now tested: umask, kernel version, domain name,
and timezone. We might only be missing CPU type and current date now.

Improvements to reproducible.debian.net
Mattia Rizzolo has been working on compressing logs using gzip to save
disk space. The web server would uncompress them on-the-fly for clients
which does not accept gzip content.
Mattia Rizzolo worked on a new page listing various
breakage: missing
or bad debbindiff output, missing build logs, unavailable build
dependencies.
Holger Levsen added a new execution environment to run debbindiff using
dependencies from testing. This is required for packages built with
GHC as the compiler only understands interfaces built by the same
version.
debbindiff development
Version 17 has been uploaded to unstable. It now supports comparing
ISO9660 images, dictzip files and should compare identical files much
faster.
Documentation update
Various small updates and fixes to the pages about
PDF produced by LaTeX,
DVI produced by
LaTeX,
static
libraries,
Javadoc,
PE
binaries,
and Epydoc.
Package reviews
Known issues have been tagged when known to be deterministic as some
might unfortunately not show up on every single build.
For example, two new issues have been identified by building with one
timezone in April and one in May. RD
and help2man add current month and
year to the documentation they are producing.
1162 packages have been removed and 774 have been added in
the past week. Most of them are the work of proper automated
investigation done by Chris West.
Summer of code
Finally, we learned that both akira and Dhole were accepted for
this Google Summer of
Code.
Let's welcome them!
They have until May 25th before coding officialy begins. Now is the good time
to help them feel more comfortable by sharing all these little bits of
knowledge on how Debian works.