Despite having been a professional Java programmer for some time, I’ve never had to use Maven until now, and to be honest I’ve avoided it. Just recently however I’ve found myself need to use it to build a pre-existing project, and so I’ve had to had to get acquainted with it to a small extent.

So what is Maven, exactly?

For some reason people who write the web pages and other documentation for build systems seem to have trouble articulating a meaningful description of what their project is, what it does and does not do, and what sets it apart from other projects. The official Maven website is no exception: the “what is Maven?” page manages to explain everything except what Maven actually is. So here’s my attempt to explain Maven, or at least my understanding of it, succintly.

Maven is a build tool, much like Ant. And like Ant, Maven builds are configured using an XML file. Whereas for Ant it is build.xml, for Maven it is pom.xml.

Maven is built around the idea that a project depends on several libraries, which fundamentally exist as jar files somewhere. To build a project, the dependencies have to be download. Maven automates this, by fetching dependencies from a Maven repository, and caching them locally.

A lot of Maven functionality is provided by plugins, which are also jar files and which can also be downloaded from a repository.

So, yeah, Maven loves downloading shit. Maven builds can fail if run offline, because the dependencies/plugins can’t be downloaded.

The pom.xml file basically contains some basic project information (title etc), a list of dependencies, and a build section with a list of plugins.

Dependencies have a scope – they might be required at test, compile time, or run time, for example.

Even basic functionality is provided by plugins. To compile Java code, you need the maven-compiler-plugin.

To Maven, “deploy” means “upload to a repository”. Notionally, when you build a new version of something with Maven, you can then upload your new version to a Maven repository.

Where Ant has build targets, Maven has “lifecycle phases”. These include compile, test, package and deploy (which are all part of the build “lifecycle”, as opposed to the clean lifecycle). Running a phase runs all prior phases in the same lifecycle first. (Maven plugins can also provide “goals” which will run during a phase, and/or which can be run explicitly in a phase via the “executions” section for the plugin in the pom.xml file).

It seems that the Maven creators are fans of “convention over configuration”. So, for instance, your Java source normally goes into a particular directory (src/main/java) and this doesn’t have to specified in the pom file.

Initial Thoughts on Maven

I can see that using Maven means you have to expend minimal effort on the build system if you have a straight-forward project structure and build process. Of course, the main issue with the “convention over configuration” approach is that it can be inflexible, actually not allowing (as opposed to just not requiring) configuration of some aspects. In the words of Maven documentation, “If you decide to use Maven, and have an unusual build structure that you cannot reorganise, you may have to forgo some features or the use of Maven altogether“. I like to call this “convention and fuck you (and your real-world requirements)”, but at least it’s pretty easy to tell if Maven is a good fit.

However, I’m not sure why a whole new build system was really needed. Ant seems, in many respects, much more powerful than Maven is (and I’m not saying I actually think Ant is good). The main feature that Maven brings to the table is the automatic dependency downloading (and sharing, though disk space is cheap enough now that I don’t know if this is really such a big deal) – probably something that could have been implemented in Ant or as a simple tool meant for use with Ant.

So, meh. It knows what it can do and it does it well, which is more than can be said for many pieces of software. It’s just not clear that its existence is really warranted.

In conclusion

There you have it – my take on Maven and a simple explanation of how it works that you can grasp without reading through a ton of documentation. Take it with a grain of salt as I’m no expert. I wish someone else had written this before me, but hopefully this will be useful to any other poor soul who has to use Maven and wants to waste as little time as possible coming to grips with the basics.

So I want a structured text format usable for configuration files and data interchange. My key requirements can be boiled down to:

Syntax allows concise expression (this alone rules out XML)

Simple to parse (this also rules out XML)

Suitable for human “consumption” (reading, editing). To some degree, this rules out XML.

As you can see, XML is definitely out. But oddly enough, I’m struggling to find anything I’m really happy with.

Oh yeah, I know, I know, JSON, right? I have these main problems with JSON:

1. Excessive quotation marks required

So for a simple set of key-values I need something like:

{
"key1" : "overquoted",
"key2" : "the perils of javascript"
}

I mean this isn’t crazy bad, but why are the quotes even necessary? I mean wouldn’t it be nice if I could instead write:

{
key1 : overquoted,
key2 : "the perils of javascript"
}

Given that, at the point key1 and key2 appear, an alphabetical character may not otherwise legitimately be present, what would be the harm in allowing this? (sure, if you want spaces or punctuation in your key, then you should need to quote it, but not otherwise). Similarly for values. It would be nice if those unnecessary quotes weren’t actually required.

2. No comments

This one really irks me. For a configuration file, comments are pretty much mandatory. Douglas Crockford gives an explanation for why there are no comments in JSON (why he removed comments from the spec, in fact), and it sucks. Basically: people weren’t using comments the way I wanted them to, so I removed comments. Yeah, I think we just hit ludicrous speed. There are so many things wrong with this argument I barely know where to begin. At the very outset, anyone using comments as parsing directives was going to need a custom parser anyway – what they were dealing with wasn’t plain JSON. So actually changing JSON does not affect those people; they will continue to use their custom parsers. In fact all you do by removing comments is make the standard less useful generally. The follow up:

Suppose you are using JSON to keep configuration files, which you would like to annotate. Go ahead and insert all the comments you like. Then pipe it through JSMin before handing it to your JSON parser.

… is equally ridiculous. I mean sure I could strip comments before handing off the the parser, but then my original data isn’t actually JSON, is it? And so interoperability is destroyed anyway, because I can no longer use any standard-JSON tools on my configuration files.

3. Unclear semantics

The current RFC for JSON has, in my opinion, a lot of guff about implementations that just doesn’t belong in a specification. Problematically, this discussion could be seen to legitimise limitations of implementations. Take for example:

An object whose names are all unique is interoperable in the sense that all software implementations receiving that object will agree on the name-value mappings. When the names within an object are not unique, the behavior of software that receives such an object is unpredictable.

What does that mean, exactly? That it’s allowed that objects with non-unique names cause software to behave “unpredictably”? This is meant to be a specification imposing requirements on implementations, but it’s hard to glean from this text precisely what the requirements are, in particular because (just above that):

The names within an object SHOULD be unique.

That’s SHOULD, not MUST (see RFC 2119); which implies that non-unique names are in fact permissible and must be handled by an implementation. I wonder how many JSON libraries will properly represent such an object… not many, I’d guess.

So How About YAML?

YAML does solve the problems that I identified with JSON above. It doesn’t require superfluous quotation marks, it’s clear about map semantics, and it allows comments. On the other hand, its specification is quite large. Part of the complexity comes from the concept of tags which are a way of identifying types. While the YAML core specification (“failsafe schema”) deals only with maps, sequences and strings, it allows for explicitly tagging any value as a particular type identified by a “tag”. The schema notionally defines how tags are mapped to actual types and allows for specifying rules for determining the type of otherwise untagged ‘plain scalar’ (roughly: unquoted string) values. So for instance the JSON schema – which makes YAML a superset of JSON – maps sequences of digits to an integer type rather than the string type. The fact that different schemas yield different semantics, and that arbitrary types (which a given implementation may not know how to handle) can be assigned to values, in my opinion reduces YAML’s value as an interchange format.

(For instance, a tool which merges two YAML maps needs to know whether 123 and “123” are the same or not. If using the failsafe schema, they are strings and are the same; if using the JSON schema, one is a number and they are not the same).

In fact, the whole notion of schemas leads to the question of whether it is really up to the text format to decide what type plain nodes really are. Certainly, maps and sequences have a distinct type and are usually unambiguous – even YAML doesn’t allow a schema to re-define those – and are enough to represent any data structure (in fact, just sequences would be enough for this). I also think it’s worth while having a standard quoting mechanism for strings, and this is necessary to be able to disambiguate scalar values from structures in some cases. But beyond that, to me it seems best just to let the application determine how to interpret each scalar string (and it can potentially use regular expressions for this, as YAML schemas do), but that for purposes of document structure scalars are always just strings. This is essentially what the YAML failsafe scheme does (and it even allows disambiguating quoted strings from unquoted strings, since the latter will be tagged with the ‘?’ unknown type).

It’s worth noting that YAML can handle recursive structures – sequences or maps that contain themselves as members either directly or indirectly. This isn’t useful for my needs but it could be for some applications. On the other hand, I imagine that it greatly complicates implementation of parsers, and could be used for attacks on poorly coded applications (it could be used to create unbounded recursion leading to stack overflow).

Or TOML?

TOML is a relative newcomer on the scene of simple structured text formats. It has a fixed set of supported types rather than allowing schemas as YAML does, and generally aims to be a simpler format; on the other hand it is much closer to YAML in syntax than JSON and so is much easier to read and edit by hand.

Among the supported types are the standard map / sequence / string, but also integer, float, boolean and date-time. This seems fine, but again I’m uncertain that having more just than the basic “string” scalar type is really necessary. On the other hand having these types properly standardised is unlikely to cause any harm.

I think the one downside to TOML is the ungainly syntax for sequences of maps – it requires double-square brackets with the name of the sequence repeated for each element:

Nested maps are also a bit verbose, requiring the parent map name to be given as a prefix to the child map name:

[parent]
[parent.child]
key1 = value1 # this key and all following keys are in the child map

The top level node of a TOML structure, if I understand correctly, must always be a map, since you specify key-value pairs. This is probably not a huge concern for my purposes but is certainly a limitation of the format. Once you’ve opened a map (“table” in TOML parlance) there’s also no way to close it, it seems, other than by opening another table.

I think the occasional ugliness of the syntax, together with the immaturity of the format, are deal breakers.

And so the winner…

… Is probably YAML, at this stage, with the failsafe schema, although the potential for recursive structures makes me a little uneasy and it’d be nicer if I didn’t have to explicitly choose a schema. It’s also a shame that the spec is so wordy and complex, but the syntax itself is nice enough I think and seems like a better fit than either JSON or TOML.

Recently I’ve tasked myself with trying to modernise my system a bit (see my previous post entitled “On Going 64-bit” for some background). Part of this is to upgrade certain base components of the system to allow better interoperability with desktop environments and programs utilising recent desktop standards. For various reasons I want to avoid installing SystemD (some background here), but that means I need to find other ways to provide various services otherwise implemented by it.

Replacements for SystemD

For my system init I’ve previously been using “SimpleInit” which used to be part of the Util-linux package (but which is no longer included in that package). More recently I’ve been toying with writing my own replacement which provides proper service management (starting and stopping services, and handling service dependencies), but that’s probably the topic of a future post; suffice to say that there are plenty of alternatives to SystemD if you need an init.

The first major piece of (non-init) functionality normally provided by SystemD is that of maintaining the /dev hierarchy (creating device nodes and various symbolic links, assigning appropriate ownership and attributes, and providing a service for other programs to be able to listen for device hotplug events). Before SystemD subsumed it, this task was performed by a daemon called Udev. There is a fork called Eudev maintained by some Gentoo folk which seems up to the task (and which is maintained), so I’m using that.

Part of what SystemD provides is seat and session management (via a service referred to as logind). In short, a seat is a set of input/output hardware usable for direct user interaction (mouse, keyboard, screen); a normal desktop computer has one seat (but you could probably plug in an extra mouse, keyboard and graphics card and call that a second seat). So “seat management” is essentially just about assigning hardware devices to a seat ID. If I log in to (create a session on) some seat then the devices belonging to that seat should be made accessible to me. Ideally you would be able to start the X server and tell it what seat to use and it would then automatically find the right keyboard, mouse and display adapter (in practice this is not possible, yet; you need to create separate configuration files for each seat).

The seat/session management aspect of SystemD actually subsumes the old ConsoleKit, which is no longer maintained. Fortunately a successor called ConsoleKit2 has been forked. ConsoleKit2 seems to me to be some way from finished, but it at least seems to run ok. ConsoleKit2 exposes, like SystemD’s logind, various functionality to manage sessions on seats, and also system shutdown/restart functions; all are exposed via D-Bus.

Finally, ConsoleKit2 (and SystemD, presumably) requires PolicyKit (apparently now called just Polkit) in order to allow policy about who can actually utilise the shutdown functionality. PolicyKit provides a D-Bus API that allows a client to query, essentially, whether a certain user can perform a certain action; the policy controlling the response is configurable. You can allow actions to be performed by certain users, or by users by belonging to certain groups, or by users who are logged in to a local seat, for instance (this latter requires ConsoleKit[2] functionality – so ConsoleKit and PolicyKit are highly coupled, but you can build ConsoleKit2 without PolicyKit and just not provide the shutdown scripts).

Ok, so let’s examine each of these components.

D-Bus

D-Bus is a socket-based service that essentially provides Remote Procedure Call (RPC) functionality. You ask for a service by name (actually, you provide a connection id or name and an object name), D-Bus launches it if it’s not running already and provides a handle, and you then send signals to / call methods on the service instance (by name, potentially with parameters that need to be marshalled to send them over the wire). It is usual to run a system bus instance and a per-session bus instance; typically, configuration allows recognized service names to be owned only by particular users (such as root).

I don’t feel like there’s anything wrong with the concept of D-Bus, but I do have some issues with the implementation:

The system instance exposes a unix socket (/var/run/dbus/system_bus_socket) with rwxrwxrwx permissions (all users can connect). To me, this seems like a local Denial Of Service attack waiting to happen.

That D-Bus handles method calls to the service seems like one step too far. Connecting to a service should just give you a stream and the protocol you use from that point should be up to the service. Allowing introspection and the ability to call methods via a script is nice, I suppose, but not necessary; it imposes a protocol with certain limitations.

Still, it could’ve been worse. At least D-Bus doesn’t use XML for the transport protocol (even though it does for its configuration files, urgh).

ConsoleKit[2] / logind

So, this is where shit starts to get messy. Basically the API provided is way too heavy for the few tasks that should really be needed. As far as use cases that I can see:

Screen savers might want to be notified if their containing session goes inactive, so they don’t waste processor cycles.

I guess the ability to request a system shutdown/restart, and to inhibit such, is useful (but why put it under the same service name as the session tracking stuff, for Pete’s sake…)

Basic session tracking; we might want to know who is logged on and to what seat, to implement an equivalent of the “who” command for instance.

I guess the ability to kill off an entire session could be useful, as could waiting for all processes in a session to finish.

Whatever basic functionality is necessary to make a determination of whether a user has a local session. PolicyKit would use this; see below.

Fast user switching functionality: see below.

This is pretty much what’s provided by SystemD (see here) or ConsoleKit2 (here). Yes, the APIs are essentially the same but for some reason the SystemD folk changed the D-Bus service name. Breaking stuff to look tough, I guess.

So-called “fast user switching” (running two or more sessions on one seat and switching between them) requires the ability to list sessions on a seat, and then the ability to activate one of these sessions, but I’m not sure that the latter functionality should be part of ConsoleKit anyway. Consider the case of Linux virtual terminals; these share a seat, and switching to another session is a matter of pressing CTRL+F1 for instance (but can also be done via an ioctl call); a user-switch applet in X (at this stage) works by just running different X sessions on different virtual terminals and switching to the appropriate virtual terminal programmatically. However, a better-designed window system might allow running multiple sessions on the same virtual terminal (consider even the case of running multiple embedded X servers such as Xephyr within a single parent X display, where switching sessions just means hiding one Xephyr window and making another visible). In this case you have a kind of virtual seat on top of a session (which is itself on top of a real seat). ConsoleKit could not be expected to know how to bring such a session to the foreground (although it might allow registering session handlers so that the session initiator can provide such a function, and in fact this seems like the perfect solution).

Note that neither ConsoleKit[2] nor SystemD allow for handling the above situation. The stupid API design means that the session manager needs to know how to switch the active session on a seat.

In terms of implementation, most of the complexity comes from the apparent desire to prevent processes from “escaping” their session (by detaching from the controlling terminal and running as a daemon, or in general by refusing to die when their parent process does). This is necessary for a reliable implementation of one of the features listed above – the ability to kill off an entire session. SystemD tries to use cgroups to group processes in a session, but it’s not clear to me that this is foolproof (since I don’t think it’s possible to atomically kill off all processes in a cgroup, although the freezer and possibly cpuset groups could be used to do that in a slightly roundabout way; the documentation implies that SystemD doesn’t do this, as it requires CONFIG_CGROUPS but “it is OK to disable all controllers”). ConsoleKit2 can also use cgroups to manage sessions, but this depends on cgmanager, and in any case ConsoleKit2 does not provide for terminating sessions (a la logind’s TerminateSession method).

So anyway, the API kind of sucks but could probably be worked into something a bit better easily enough. Also, ConsoleKit2 needs some work before it provides all the session management functionality of SystemD. Which brings us to the final piece of the puzzle.

PolicyKit (Polkit)

I wasn’t even going to bother with Pol[icy]Kit, but ConsoleKit2 gives you a nasty (if ultimately rather meaningless) warning at build time if you don’t have it, and anyway it would be nice if the “shutdown” functionality of my desktop actually worked. The principle behind Polkit is pretty straightforward: user tries to invoke a command on a service (whether by D-Bus or some other means); the service asks Polkit if the user (apparently identified by any of session, process, and/or user id) is allowed to invoke that command; Polkit checks its configuration and replies yay or nay. Sometimes Polkit wants authentication (a password) first, and if so it uses a D-Bus object that has been registered to act as an Authentication Agent for the session (presumably, it checks to make sure that the process registering the agent for a session actually belongs to that session, otherwise there’s a security hole).

Polkit can give special privilege to local processes, that is, processes in a session on a local seat. This might be used for instance to allow local users to shut down the system, or mount USB devices. In general, then, Polkit makes a decision based on:

The user id of the process invoking the request (in case it is not provided by the service)

Whether that process is part of a local session

In some cases, whether the authentication via the provided agent succeeds.

For all this to work, Polkit needs:

to be able to determine the session ID of the process invoking the request (so that it can check if the session is local)

to be able to check if a session is local

to be able to determine the user ID of the process invoking the request (if it was not provided by the service)

Both of these requisite functions are provided by ConsoleKit/Logind. I have one issue with the concept, which is that I think determination of being “local” should be made solely based on the user id and not the process. That is, if a user has a local session, than any process belonging to that user in any session (including a non-local session) should be treated as local. Putting it another way, users and not processes should have privilege. This would simplify the requirements a little; the list of three items above because a list of just two. However this is really a minor quibble, I guess.

No, my biggest complaint about Polkit is that you have to write the policies in fucking Javascript. Yes, Polkit actually has a javascript interpreter as a dependency. Considering how the factors above are most likely the only ones relevant in the policy this seems like a case of excessive bloat. Also, it’s Javascript, a language which doesn’t even properly support integers. (Now would be a good time to go watch the Wat talk, if you haven’t already, I guess).

Do I really have to install this piece of filth just so I can reboot my desktop using a GUI?

Just recently I wrote about the clever “dual ABI” hack found in GCC 5’s implementation of the standard C++ library. I quipped at the end of that post:

I suspect the right way to handle changing ABI is to do it the way it’s always been done – by bumping the soname.

Having recently done some work on a C++ project, I’ve discovered an issue present in the dual ABI implementation. What seemed like trivial code to handle an exception from attempting to open a non-existent file just wasn’t working. I worked it down to the following test case:

Surprisingly, this fails to work properly (prints the wrong message) when compiled with a straight “g++ -o simple simple.cc”. It works correctly when compiled instead with “g++ -D_GLIBCXX_USE_CXX11_ABI=0 -o simple simple.cc”. I would have filed a bug, but it seems the problem is known.

This is a pretty major bug. Any C++ program that handles I/O errors by catching the appropriate exceptions just isn’t going to work. Furthermore, this isn’t just a minor problem; having to separate out implementations of iosbase and derived classes for the two ABIs would significantly reduce the benefit of even having the dual ABI. The problem with “clever” hacks like the dual ABI libstdc++ is that they are (1) often not so clever and are (2) always most definitely hacks. The proposed solution? Heap more hacks on top so the first hack works:

One option to make both forms work would be to hack the EH runtime to make handlers for std::ios_base::failure able to catch std::ios_base::failure[abi:cxx11] objects, and vice versa, creating an object of the other type on the fly.

Urgh. Just urgh.

What really annoys me most about this nonsense is that the one sane option I have – stripping out the ‘abi_tag’ nonsense from the source, re-compiling libstdc++ and calling it libstdc++.so.7 – would probably cause me problems further down the track, because third-party binaries are going to want the new ABI with the old soname (and even if I didn’t care about those, I’d still be one version number up from the official libstdc++ from that point on, which feels like it could cause problems).

My desktop system almost exclusively runs software that I compiled from source. It’s been that way for a long time, since I got frustrated with the lacklustre pace of Debian-stable software updates and with the package management system in general (this was 1998 or so). I began compiling packages from source rather than updating them via the Debian repository, and eventually decided to purge the remaining parts of Debian from my system, replacing Debian packages with compiled-from-source counterparts.

What can I say; I was young and optimistic. Surprisingly enough, I managed to get a working system, though compiling many packages was fraught with unwarranted difficulty and required various hacks to makefiles and buildscripts. Linux From Scratch didn’t even exist yet, so I was mostly on my own when I ran into problems, but I persevered and it payed off. I had a nice, fast, working system and I had a good understanding of how it all fit together. Later, when Fedora and Ubuntu appeared on the scene (and Debian had got its act together somewhat) I felt no desire to switch to these fancy new distributions because doing so would mean losing the strong connection with the system that I had pieced together by hand.

Sure, I faced some problems. Upgrading packages was a difficult, and often required upgrading several of the dependencies. Uninstalling packages was a nightmarish procedure of identifying which files belonged to the package and deleting them one by one. I learned that I could use “make DESTDIR=… install” to output many packages into their own root, and I eventually I wrote a shell script that would “install” packages by symbolically linking them into the root file system from a package specific root, and could uninstall them by removing those links – so I had a kind of rudimentary package management (I cursed those packages which didn’t support DESTDIR or an equivalent; I often ended up needing to edit the makefile by hand). I usually compiled without many of the optional dependencies and to a large extent I avoided the “dependency hell” that had plagued me while using Debian. I found Linux From Scratch at some point and used its guides as a starting point when I wanted to upgrade or install a new package. I kept notes of build options and any specific hacks or patches that I had used. Larger packages (the Mozilla suite and OpenOffice come to mind) were the most problematic, often incorporating non-standard build systems and having undocumented dependencies; to fix an obscure build problem I often had to tinker and then repeat the build process, which could run for hours, multiple times until I had managed to work around the issue (anyone who’s read this blog is now probably starting to understand why I have such a loathing for bad build documentation!).

Despite all the problems and the work involved, I maintained this system to the present day.

Modernising

When I started building my system, the processor was a 32-bit Pentium III. When an upgrade gave me a processor that was 64-bit capable, it didn’t seem like the effort of switching to the new architecture was justifiable, so I stuck with the 32-bit software system. Recently I decided that it’s time to change, so I begun the task of re-compiling the system for the 64-bit architecture. This required building GCC as a cross-compiler, that is, a compiler that runs on one architecture but that targets another.

Building a cross-compiler was not as easy as I had hope it would be. GCC requires a toolchain (linker, assembler etc) that supports the target architecture. GNU Binutils targeting on a 32-bit architecture cannot handle the production of 64-bit binaries, so the first step (and probably the easiest) was to build a cross-target Binutils. This was really about as simple as:

However, building GCC as a cross compiler is nowhere near as trivial. The issue is that GCC includes both a compiler and a runtime-support library. Building the runtime-support library requires linking against an appropriate system C library, and of course I didn’t have one of those, and I couldn’t build one because I didn’t have a suitable cross-compiler. It turns out, however, that you can build just enough of GCC to be able to build just enough of Glibc that then you can then build a bit more of GCC and then the rest of Glibc and finally the rest of GCC. This process isn’t formally documented (which is a shame) but there’s a good rundown of it in a guide by Jeff Preshing, without which I’m not sure I would have succeeded. (Some additional notes on cross compiling GCC are included at the end of this post).

Now I had a working cross-compiler targeting the x86_64 platform. When built as a cross-compiler GCC and Binutils name their executables beginning with an architecture prefix, so for instance “gcc” becomes “x86_64-pc-linux-gnu-gcc” and “ld” becomes “x86_64-pc-linux-gnu-ld” (you actually get these names when building as a non-cross compiler, too, but in that case you also get the non-prefixed name installed). To build a 64-bit kernel, you simply supply some variables to the kernel make process:

make ARCH=x86_64 CROSS_COMPILE="/opt/x86_64/bin/x86_64-pc-linux-gnu-" menuconfig
make ARCH=x86_64 CROSS_COMPILE="/opt/x86_64/bin/x86_64-pc-linux-gnu-" bzImage
# and so on

The “CROSS_COMPILE” variable is the prefix used for any compiler/binutil utility when producing object for the target. So for example “gcc” is prefixed to become “/opt/x86_64/bin/x86_64-pc-linux-gnu-gcc”.

I built the kernel, and booted to it. It worked! … except that it refused to run “init”, because I hadn’t enabled “IA-32 emulation” (the ability to run 32-bit executables on a 64-bit kernel). That was easily resolved, however.

I then set about building some more 64-bit packages: build Binutils as a 64-bit native package, re-building Glibc without the special –prefix, building GCC as a 64-bit native compiler (which required first cross-compiling its prequisites, including MPFR, GMP and MPC). All seems to be going ok so far.

Multi-lib / Multi-arch

One issue with having 32-bit and 64-bit libraries in the same system is that you have name clashes. If I have a 32-bit /usr/lib/libgmp.so.10.1.3, where do I put the 64-bit version? It seems that the prevalent solution is to put 64-bit libraries in /usr/lib64 (or /lib64) and leave only 32-bit libraries in plain /usr/lib and /lib. The Glibc dynamic linker hard-codes these paths when compiled for x86_64, it seems. So, when I build 64-bit packages I’ll use –libdir=/usr/lib64, I guess, but I don’t like this much; the division seems pretty unnatural, mainly in that it favours the 32-bit architecture and is somewhat arbitrary. Debian and Ubuntu bothappear to have similar reservations and are working on “Multiarch spec”, but for now I’ll go with the lib/lib64 division as it’s going to be less immediate hassle.

I also still have the issue that I can’t control which packages pkg-config will detect. I guess I can use the PKG_CONFIG_PATH environment variable to add the 64-bit paths in when doing a 64-bit build, but I can’t prevent it from picking up 32-bit packages in the case where the 64-bit package isn’t installed, which I imagine could lead to some pretty funky build errors.

That’s about where I’m at. It wasn’t particularly easy, but it is done and it seems to work.

If it’s not clear from Jeff’s guide, –prefix for your GCC cross-compiler build should be /xyz (or whatever you like) and –prefix for Glibc should be /xyz/$TARGET, in my case I used /usr/x86-64 and /usr/x86-64/x86_64-pc-linux-gnu. The GCC cross-compiler will expect to find libraries and include files in the latter (under …/lib and …/include).

I wanted multilib support, whereas Jeff’s guide builds GCC without multilib. For this you need the 32-bit Glibc available to the cross-compiler, in $GLIBC_PREFIX/lib/32 (being /usr/x86-64/x86_64-pc-linux-gnu/lib/32 in my case). I symlinked various libraries, but you might get away with just symlinking your entire /usr/lib as $GLIBC_PREFIX/lib/32. You also need $GLIBC_PREFIX/include/gnu/stubs-32.h (which you can link from /usr/include/gnu/stubs-32.h).

During the Glibc build I got an error about an unresolved symbol, on something like __stack_chk_guard (foolishly I did not record the exact error). I got around this by configuring Glibc with ‘libc_cv_ssp=no’.

If you install 64-bit libraries into the standard /usr/lib and want to link against them when building with your cross-compiler, you’ll need to add -L/usr/lib to your linker flags (eg LDFLAGS=-L/usr/lib during configure).

With GCC 5.2 just having been released, I figured it was time to upgrade my system to the latest-and-greatest in GNU compiler technology. So far the new compiler seems fine, but there’s a subtle issue that has been introduced to do with the “dual ABI” in the standard C++ library implementation, libstdc++. A Red Hat developer (“rhjason”) writes about it here. If I try to boil it down to the essence:

The C++11 standard has some complexity requirements which require re-engineered implementations of certain standard classes, among them std::string and std::list. This thereby requires an ABI change (perhaps the sizes of the structures have changed, or inline functions exist which now need to be implemented differently; given that std::list is a template, there’s really no getting around this).

“Complexity requirements” refers to the algorithmic complexity (think ‘big-O’ notation). In particular std::list size() must now be O(1) – that is, it must take a constant amount of time regardless of the number of elements in the list (GCC bug). Previously the size() method worked by counting the elements in the list, which is O(n); there’s an explanation (in the form of an argument against) here. (I’m not sure if I agree with this argument).

Changing the ABI normally means changing the soname, the name of the dynamic library that you link against. In this case it would have been a bump from libstdc++.so.6 to libstdc++.so.7.

This does, admittedly, come with problems. It’s possible to have some program (A) link against a library (B) and also against the C++ library (libstdc++.so.7). However, it may be the case that the library B was linked against the older version of the C++ library (libstdc++.so.6). When you execute A, then, it will dynamically link against two versions of the library. Because these libraries have (at least some) common symbol names, one will override the other. That means, for instance, that the library B will call some function but have the libstdc++.so.7 execute, which has the wrong ABI. There’s also the possibility of passing standard library objects (such as strings) between A and B, for which the ABI differs. Either case might lead to data structure corruption, incorrect behavior and possibly crashing.

(Note that this problem is not limited to the C++ library. Any library which changes its ABI and bumps its soname potentially causes similar issues).

To combat these problems, the GCC/libstdc++ folk decided to put the old and new ABI into a single dynamic library (libstdc++.so.6).

In many cases, nested inline namespaces (explanation) are used to avoid symbol clashes between the old and new ABI; however this is not possible in every case (because you can’t have a namespace inside a class, for instance). So, the compiler now understands ‘__attribute (( abi_tag(“cxx11”) ))’, which can be applied to an inline namespace or a declaration.

(I do not understand why the tag should ever actually need to be applied to a namespace, and indeed it’s not really done in libstdc++. It seems that you do not need to supply the abi argument if you do – so you can have just ‘__attribute ((abi_tag))’ – but neither is it at all clear to me what the purpose of this would be. With a few experiments I see that it has some effect on warnings when -Wabi-tag is used, and might affect the abi tag that functions/variables inside the namespace “inherit” if they use an abi-tagged type).

You can select the ABI to compile against using the _GLIBCXX_USE_CXX11_ABI preprocessor macro (set it to 1 for the new ABI, or 0 for the old ABI). If the macro is not set it will be set to 1 when you include a C++ header.

So, this allows two ABIs to exist in a single dynamic library. As an added bonus you may get link-time errors if you try to link to a library using the other ABI. However, there is at least one significant issue with this strategy; Allan McRae (an Arch Linux guy) writes about it here:

This discovered an issue when building software using the new C++ ABI with clang, which builds against the new ABI (as instructed in the GCC header file), but does not know about the abi_tag attribute. This results in problems such as (for example) any function in a library with a std::string return type will be mangled with a [abi:cxx11] ABI tag. Clang does not handle these ABI tags, so will not add the tag to the mangled name and then linking will fail.

In other words, the GCC people have basically decided that other compilers don’t exist. This is, I have to say, pretty shitty. The LLVM guys are now forced to choose between supporting this non-standard abi_tag attribute or supporting only the non-C++11 ABI when using GNU libstdc++. Although I hope they go with the former (because it’ll make life easier for me personally) one could understand if they decided not to. I suspect the right way to handle changing ABI is to do it the way it’s always been done – by bumping the soname.

I think that compiler bugs – the kind where they produce the wrong code, i.e. an incorrect compilation – are perhaps the worst kind of bug, because they can be very difficult to identify and they can cause subtle issues (including security issues) in other code that should work correctly. That’s why this GCC bug bothers me a lot. Not only is it a “wrong code” bug, but it is easy to reproduce without using any language extensions or unusual compiler options.

It’s easy to reproduce, and it’s obviously wrong. Somehow the compiler is managing to mess up the calculation of (2 * 2 + 6 * 5). And yet, it’s classified as Priority 2. And furthermore, GCC 4.9.3 was just recently released, with this bug still present. It makes me start to wonder about the quality of GCC. I’m waiting for this to be fixed before I move to the 4.9 series (5.x series is way too new for my liking, though I might skip over 4.9 if 5.2 is released in the near future).

I’d like to run my compiler benchmarking tests on LLVM 3.6.1 and GCC, but I’ll hold off for a bit. I have done some quick testing with LLVM though and I have to say it is exceeding expectations.