This page is intended for developers making changes to the PNaCl tools. If you just want to build PNaCl-based applications, see the Native Client developer documentation. If you just want to build the toolchain, e.g. to package PNaCl or Chromium with NaCl support, see the shorter document on building.

Prerequisites

In addition to what is needed to [build nacl], several packages may need to be installed on your system. For Ubuntu systems, e.g.:

sudo apt-get install cmake texinfo flex bisong++-arm-linux-gnueabihf

Get the Native Client Source

To build the PNaCl toolchain, you will need to check out the entire native client project with git. After that:

If you want to use these binaries outside of Scons, you must build and install the SDK libraries. This installs the libraries such as libnacl from the NaCl repo that go into the SDK and are needed to link programs without any special flags. (Note that this does not install the Pepper libraries, as those come from the Chromium repository):

pnacl/build.sh sdk

Sanity Check: Compiling a test program with the prebuilt toolchain

At this point, you probably want to test if the pre-built toolchain works. Let's start by creating a "hello world" program:

#include <stdio.h>

int main(int argc, const char *argv[]) { printf("Hello World!\n");

return 0;

}

Add the PNaCl toolchain to your path for convenience, if desired:

PATH=${PATH}:toolchain/linux_x86/pnacl_newlib/bin # Adjust the name for your system

Now compile hello.c:

pnacl-clang hello.c -o hello.nonfinal.pexe

pnacl-finalize hello.nonfinal.pexe -o hello.pexe

(NOTE: If you get an error about missing crt1.o or -lnacl, you must run the SDK install step. See the installation instructions in the previous section.)

You have now produced a "pexe", a PNaCl bitcode application. To run this pexe, it must be first translated to native code. Let's translate it to X86-32, X86-64 and ARM.

pnacl-translate -arch x86-32 hello.pexe -o hello_x86_32.nexe

pnacl-translate -arch x86-64 hello.pexe -o hello_x86_64.nexe

pnacl-translate -arch arm hello.pexe -o hello_arm.nexe

Now you have 3 native executables, one for each architecture. To run them inside of sel_ldr, use the helper script, native_client/run.py:

./run.py hello_x86_32.nexe

./run.py hello_x86_64.nexe

./run.py hello_arm.nexe # By default it will look for qemu to run the ARM nexe.

Testing a PNaCl toolchain with SCons

To use the PNaCl toolchain with the native client scons tests, just specify "bitcode=1" on the command-line.

pnacl/test.sh test-x86-32-sbtc # Runs the same tests, but with the sandboxed translator

pnacl/test.sh test-x86-64-sbtc

pnacl/test.sh test-arm-sbtc

Other Compiler Tests

LLVM regression tests

These tests can be run directly by running 'make check' in the build directory (pnacl/build/llvm_x86_64). There are currently a few known failures which are not marked as XFAIL in the usual LLVM way, so instead the regression tests can also be run in the following way, which will ignore the known failures. (This only works after you have built the PNaCl LLVM and have the build directory available).

pnacl/scripts/llvm-test.py --llvm-regression

The LLVM regression tests run quickly and should be run before every LLVM commit, along with the scons tests.

The following tests take longer and are generally not run by the developers on every commit, but they should be run for large changes, and are run on the toolchain testing bots on every commit.

GCC Torture tests

To run the gcc torture test suite against PNaCl, do the following:

tools/toolchain_tester/torture_test.py pnacl x86-64 --concurrency=12

Other options are documented in torture_test.py and tools/toolchain_tester/toolchain_tester.py

LLVM Test Suite

To run the LLVM test suite against PNaCl, do

pnacl/scripts/llvm-test.py --testsuite-all --arch x86-64

This is equivalent to

pnacl/scripts/llvm-test.py --testsuite-prereq --arch x86-64

pnacl/scripts/llvm-test.py --testsuite-clean

pnacl/scripts/llvm-test.py --testsuite-configure

pnacl/scripts/llvm-test.py --testsuite-run --arch x86-64

pnacl/scripts/llvm-test.py --testsuite-report --arch x86-64

The test suite needs to be cleaned and re-configured each time you switch architectures, but the results files persist after cleaning, so testsuite-report will give the result from the last run for that architecture. Because there is no good way to filter the set of tests that are actually run and there are a lot of known failures (e.g. tests that have not been ported to use newlib), this test suite takes a long time to run.

You need to set LD_LIBRARY_PATH before running the test suite, otherwise the native build of each test, whose output the PNaCl build is validated against, will fail to run. It's also helpful to set PNACL_CONCURRENCY.

Spec2k

If you have access to the spec2k benchmark suite you can use the harness in tests/spec2k/ to run some more extensive tests.

If SPEC_DIR is the directory containing the benchmark code/data, run:

pnacl/test.sh test-spec <SPEC_DIR> SetupPnaclX8664Opt

test.sh is a convenience script and clobbers old results, etc. The actual steps are:

pushd tests/spec2k

./run_all.sh CleanBenchmarks

./run_all.sh PopulateFromSpecHarness <SPEC_DIR>

./run_all.sh TimedBuildAndRunBenchmarks SETUP

popd

where SETUP is for example "SetupPnaclArmOpt". See "run_all.sh" for a full list of setups.

Subzero standalone tests

These tests depend on having built PNaCl's branch of LLVM from source (uses some of the headers, etc.). See "checking out PNaCl sources, building and testing" below.

In general look at the Makefile.standalone file for various targets and options. E.g., you can compile with DEBUG=1 while debugging, so that variables aren't optimized-out and functions aren't so inlined, etc.

cd toolchain_build/src/subzero

make -j32 -f Makefile.standalone check

Toolchain Development

TL;DR for checking out PNaCl sources, building and testing

If you are using Chromium's build of clang to build PNaCl (the default) and you have never done so, you need to install it.

From the native_client/ directory:

../tools/clang/scripts/update.sh

From now on, when you run 'gclient sync' clang will also be updated if needed.

--install=INSTALL After building, copy contents of build packages to the

specified directory

Sandboxed translator build

The build.sh script is now only used to build the sandboxed translator. It has 2 relevant arguments:

translator-all: builds the sandboxed translator nexes (not included in 'build-all' or 'all')

clean: cleans the build files

However, it is called from toolchain_build_pnacl.py for the bots, and it expects to use the toolchain built by the translator_compiler target of toolchain_build_pnacl.py. Use the --build-sbtc flag of toolchain_build_pnacl.py and build everything (or just the sandboxed_translators target and its dependencies) for that.

pnacl/test.sh

This script runs toolchain tests. It deletes old testing artifacts before running. The main invocation is test-all.

pnacl/driver/*

This directory contains the compiler drivers, which are in Python. If you change them, run e.g. "toolchain_build/toolchain_build_pnacl.py --install toolchain/linux_x86/pnacl_newlib driver_i686_linux" to install them.

Git Repositories

Most of the code required to build the PNaCl toolchain lives in git repositories hosted on chromium.googlesource.com. The build.sh script will automatically check out and update these repositories. The checked-out code is placed in toolchain_build/src.

Relevant Directories

toolchain_build/src

holds all the git checkouts managed by toolchain_build_pnacl.py

toolchain/${os}_x86/pnacl_newlib

root of PNaCl toolchain installed by package_version.py

toolchain_build/out

holds build directories for the pnacl toolchain

toolchain_build/out/toolchain_build.log

holds the logs of the builds

Process for making changes to the PNaCl toolchain

If you've made changes to PNaCl toolchain, either to the driver or git repositories, you can rebuild PNaCl by doing:

The first time you set up code review you'll need to let git cl do some setup:

cd toolchain_build/src/llvmgit cl config

Just press Enter at the prompts.

To do local development, create a new branch:

git checkout -b mybranch origin/master

The last argument to git checkout (the start point) is important because if your local branch does not track the remote master branch, then git cl land will not work)

Edit your code, and commit as many times as you want:

git commit -a -m “comment describing changes”

Then use git-cl to upload to the code review site:

git cl upload

You'll need to authenticate to codereview.chromium.org also. Most likely this will be your ldap email. This will upload the review, and give you an issue number, e.g. http://codereview.chromium.org/1557002. Go to the issue webpage, and edit the change summary, add reviewers, then hit send mail. Command line options from gcl such as reviewers ("-r") and send mail ("--send_mail") are also available with git cl.

Git cl associates each branch with a review issue ID. So you if you git cl upload again from the same branch, it will upload a new patch set to the same issue.

When you get the LGTM, you can push the changes.

git cl land

If there have been changes on the master branch since you forked your local branch, git cl may complain, in which case you should rebase it against the new HEAD before you push:

At this point the official toolchain builder is still not incorporating your changes as it builds off of specific commit IDs in the external repo.

To fix this modify native_client/pnacl/COMPONENT_REVISIONS to point at your new commit ID which look like

llvm=efa7b3d5c85f0242787eb091dadedc2727215df4

You can then make a standard native_client change list and test the CL with trybots (CLs affecting pnacl/ will end up selecting pnacl trybots). Upload, trybot, get an LGTM, then commit. You can also test locally by cleaning the toolchain directory, rebuilding, and testing as described earlier.

Or, you can automatically update the file using a script: pnacl/deps_update.py

Now create a CL and test it with some try jobs (e.g., git cl try my_cl_name). NOTE: You will need to wait for a toolchain to be built (see the waterfall) before this change actually works (downloads a toolchain tarball). Also see build/update_pnacl_tool_revisions.py to automate this process.

Misc Notes

Building Validators

In the case that you want to run the native client validators on your nexes, build the validators like so:

./scons targetplatform=arm sdl=none arm-ncval-core

./scons platform=x86-32 sdl=none ncval

./scons platform=x86-64 sdl=none ncval

The validators will be placed in scons-out/{host}-{platform}/staging/.