-Werror is hardcoded in the configure script, which is a very bad idea, and the opposite of portable.
using -Werror is a guarantueed build break whenever the build is tried on a system the original developer had no access to.
it’s sufficient to use a different compiler version, different libc version, etc to make new warnings pop up.

fixed with
sed -i 's/-Werror//' configure

obstacle 2 – unconditional inclusion of internal glibc header

compat/issetugid_linux.c:7:30: fatal error: gnu/libc-version.h: No such file or directory

many people assume linux == glibc, but that is not the reality.
sabotage linux uses musl libc, and there are at least 4 other libcs that could be used instead (uclibc, dietlibc, klibc, bionic).

looking at issetugid_linux.c uncovers a dubious hack:
if glibc 2.19 is detected, getauxval(AT_SECURE) is not used, because there was once a bug (see comment in source code).

however it’s common practice in distros to backport bugfixes, without updating the version number.
so this hack prevents proper usage of getauxval even if your libc version is long fixed.
the mentioned bug is very likely already fixed in any distro using glibc 2.19.

to get the thing out of my way and compilation going on, the quick fix was to cover everything with #ifdef __GLIBC__.
what the code really should do though is to just use the getauxval call unconditionally without the glibc version check.

sysctl does not work, and NEVER worked. using it is bogus.
it was a bogus experimental syscall that was deprecated before it was ever used (basically, a broken binary version of /proc/sys, without any stability between kernel versions for what the binary constants meant).

since the code in question does not use the sysctl function (declared in sys/sysctl.h) and does the syscall() directly,
it was safe and sufficient to just remove the include statement.

on the plus side: using 8 cores, libressl builds in about 1 minute, while openssl requires 1:45.
also openssl depends on perl, which takes an additional 2 minutes buildtime.
so if nothing else depends on perl, it’s about 3x faster.

compatibility

with libressl in place, a “world” metapackage (contains almost all packages) build was started.
the results:

most of the methods used in this file to gather entropy are very dubious.
the crypto experts from OpenBSD should know better and just use /dev/urandom and/or getauxval(AT_RANDOM)
instead of all these hacks.

the last build error was in apache:
ssl_engine_init.c:445:28: error: ‘ENGINE_CTRL_CHIL_SET_FORKCHECK’ undeclared

this is a macro which is available in openssl’s engine.h, and was removed from libressl for unknown reasons.
not patched yet.

apart from these two, everything seems to be usable without big effort.
so if the libressl developers rip out all their dubious entropy generation methods in favor of /dev/urandom on linux it might be well worth switching to it.

OpenBSD released an updated version 2.0.1 earlier today.
the new release fixes the following problems
– reference to main() which breaks packages using -fvisibility=hidden
– usage of -Werror
– generation of pkg-config files
– unconditional inclusion of sys/sysctl.h

i always wondered though, how such a small or “lite” package can take such a considerable amount of time to build.

as the main author of the sabotage linux distribution, building software is my daily bread, so i own a pretty fast build box.
it’s an 8 core machine with 3.1 GHz, which builds a complete 3.11 linux kernel in less than 5 minutes, making use of all 8 cores via the nice parallel build feature of GNU make.
make -j8

when invoking make like this, it first determines the dependencies between the translation units,
and then runs up to 8 build processes, one per cpu core, each one building a different .c file.

how is it possible that postgresql, shipping 16 MB of compressed sources, as opposed to 1.8MB of sqlite, builds 33% faster ?

if you look at the user times above, you start getting an idea.
the user time (i.e. the entire cpu time burnt in userspace) for postgresql is 1m53, while the total time that actually passed was only 36s.

that means that the total work of 113 seconds was distributed among multiple cpu cores.
dividing the user time through the real time gives us a concurrency factor of 3.13.
not perfect, given that make was invoked with -j8, but much better than sqlite, which apparently only used a single core.

so in the top level builddir, there are just 2 C files, one being 90 KB, and the other roughly 5MB.
the 90KB version is built in less than 1 second, so after that the entire time spent is waiting for the single cpu core building the huge sqlite3.c.

so why on earth would somebody stuff all source code into a single translation unit and thereby defeat
makefile parallellism ?

after all, the IT industry’s mantra of the last 10 years was “parallellism, parallellism, and even more parallellism”.

In addition to making SQLite easier to incorporate into other projects, the amalgamation also makes it run faster.
Many compilers are able to do additional optimizations on code when it is contained with in a single translation unit such as it is in the amalgamation.

so they have 2 reasons for wasting our time:

reason 1: easier to incorporate

reason 2: generated code is better as the compiler sees all code at once.

let’s look at reason 1:
what they mean with incorporation is embedding the sqlite source code into another projects source tree.

it is usually considered bad practice to embed third-party source code into your own source tree, for multiple reasons:

every program that uses its own embedded copy of library X does not benefit from security updates when the default library install is updated.

we have multiple different versions on the harddrive and loaded in RAM, wasting system resources

having multiple incompatible versions can lead to a lot of breakage when it’s used from another lib:
for example application X uses lib Y and lib Z, and lib Z uses a “incorporated” version of lib Y.
so we have a nice clash of 2 different lib Y versions. if lib Y has global state, it will get even worse.

if the library in question is using some unportable constructs, wrong ifdefs etc., it needs to be patched to build.
having to apply and maintain different sets of patches against multiple different versions “incorporated” into other packages, represents a big burden for the packager.

instead, the installed version of libraries should be used.

pkg-config can be used to query existence, as well as CFLAGS and LDFLAGS needed to build against the installed version of the library. if the required library is not installed or too old, just throw an error at configure time and tell the user to install it via apt-get or whatever.

conclusion: “incorporation” of source code is a bad idea to begin with.

now let’s look at reason 2 (better optimized code):
it possibly sometimes made sense to help the compiler do its job in the 70ies, when everything started.
however, it’s 2013 now.
compilers do a great job optimizing, and they get better at it every day.

since GCC 4.5 was released in 2010, it ships with a feature called LTO
it builds object files together with metadata that allows it to strip off unneeded functions and variables, inline functions that are only called once or twice, etc at link time – pretty much anything the sqlite devs want to achieve, and probably even more than that.

conclusion: pseudo-optimizing C code by stuffing everything into a big file is obsolete since LTO is widely available.
LTO does a better job anyway – not that it matters much, as sqlite spends most time waiting for I/O.
every user who wants to make sqlite run faster, can simply add -flto to his CFLAGS.
there’s no need to dictate him which optimization he wants to apply.
following this logic, they could as well just ship generated assembly code…

but hey – we have the choice !here ‘s actually a tarball containing the ORIGINAL, UN-ANAL-GAMATED SOURCE CODE…

sqlite-src-3080002.zip (5.12 MiB)
A ZIP archive of the complete source tree for SQLite version 3.8.0.2 as extracted from the version control system.
The Makefile and configure script in this tarball are not supported.Their use is not recommended.
The SQLite developers do not use them.
You should not use them either.
If you want a configure script and an automated build, use either the amalgamation tarball or TEA tarball instead of this one.
To build from this tarball, hand-edit one of the template Makefiles in the root directory of the tarball and build using your own customized Makefile.

Note how the text talks about “this tarball” despite it being a ZIP file.

Fun. there’s only a single TARball on the entire site, so that’s what you naturally pick for your build system.
and that one contains the ANAL version.
Note that my distro’s build system does not even support zip files, as i don’t have a single package in my repo that’s not building from a tarball.
should i change it and write special case code for one single package which doesn’t play by the rules ?
i really don’t think so.

funny fact: they even distribute LINUX BINARY downloads as .zip.
i wonder in which world they live in.

why do i care so much about build time ? it’s just a minute after all.
because the distribution gets built over and over again. and it’s not just me building it, but a lot of other people as well – so the cumulated time spent waiting for sqlite to finish building its 5 MB file gets bigger and bigger each day.
in the past i built sqlite more than 200 times, so my personal cumulated wasted time on it already exceeds the amount of time i needed to write this blog post.

so what i hope to see is sqlite

using tarballs for all their sources (and eventually distribute an additional .zip for windows lusers)

using tarballs for their precompiled linux downloads

either getting rid of the anal version entirely, now that they learned about LTO, or offer the anal version as an additional download and do not discourage users from using the sane version.