Running x86_64/amd64 executables on PPC64le IBM Power 9 Gentoo

Running android-studio on a Talos II ppc64 machine requires the aapt2 binary. That is currently only a part of the x86_64 build-tools, so I had to learn a way to run x86_64 executables on ppc64 cpu. The first option is full emulation (emulating x86_64 kernel and apps), the other is QEMU’s user mode translation (emulating only user-mode, translating syscalls to a running ppc64 kernel). Let’s have a quick look at the QEMU user-mode translation.

It is described and demoed (under chroot though) in short at Raptor’s website. I don’t have enough time to describe every single step so this article contains just a couple of hints which I missed when I was attempting to do this.

I have a Gentoo ppc64le system compiled fully from source. I made it using Catalyst on x86 and various profile dirty hacks, then copied to Talos HDD – I don’t remember exact steps but I can tell you it took approximately 4 days to figure out and it is possible to achieve (for the compilation of amdgpu support and gnome or other graphical stuff you should reserve couple more days and be prepared to manually modify a few ebuilds).

Once you have a working Gentoo ppc64 installation, the next step is to get an x86_64 cross-dev environment if you don’t have it already from Catalyst preparation. It will create a directory /usr/x86_64-multilib-linux-gnu/ and it is made by cross-x86_64-multilib-linux-gnu/glibc and similar ebuilds.

Then we need QEMU with proper targets (add this to make.conf and emerge qemu):

QEMU_USER_TARGETS="x86_64"

To allow running x86_64 apps like any other app, binfmt_misc (CONFIG_BINFMT_MISC) needs to be enabled in the kernel and x86_64 ELF definition loaded. Either look at /etc/init.d/qemu-binfmt (Open-RC) or add this file /etc/binfmt.d/qemu-x86_64.conf and enable systemd-binfmt service (for systemd):

This will allow running x86_64 binaries directly using qemu-x86_64 automatically.

But unless you use chroot, it still won’t work because of a missing ld linker and libraries. Using strace you can see that it tries to load libraries under various paths. It searches for a linker at /lib64/ld-linux-x86-64.so.2 so we have to make a symlink there. For other libraries it tries /lib64/x86_64 and /usr/lib64/x86_64 which are perfect candidates for symlinking. The only remaining path is gcc libs, we can’t add them to /etc/ld.so.conf.d where your ppc64 GCC LDPATH is, so we have to make symlinks for individual files. In other words you want to do something like this:

After that, x86 binaries should auto-magically work just like any other binary, without using an explicit chroot.

You may come across a problem that an executable fails to load libraries despite you see that the x86_64 library is found. All you see is mmap fail with errno EFAULT and errors like “error while loading shared libraries” or “failed to map segment from shared object”. That is because your kernel is not compiled with 4k page size! In kernel menuconfig, it is under “Kernel config” -> “Page size” (CONFIG_PPC_4K_PAGES=y). It has to be 4k, because it is x86_64 standard page size.