Post navigation

Compile And Patch Your Own Secure Linux Kernel With PaX and Grsecurity

The reason why Linux and open source will always be more secure than Windows is because you can build advanced security right in. This guide’s going to show you how to do that. I’ll be explaining how to compile your own kernel with PaX and Grsecurity patches. (I’m actually compiling 3.3.6 for myself but this guide covers 3.2.18.)

First let’s explain a few things…

What Is A Kernel?

Well, if you don’t know what a kernel is this guide is probably a bit too advanced but far be it from me to turn you away. The kernel is essentially the core of the operating system. It is the lowest level of software on the system running with the highest rights. Linux runs on “The Linux Kernel” and that kernel is common to all Linux distributions.

What Is PaX?

PaX is a series of patches that can be applied to the Linux kernel. These patches provide functionality and improvements to built in features such as ASLR (Address Space Layout Randomization), which help mitigate and deter exploits.

PaX features can be daunting so for a per-feature explanation see this link. I highly suggest you take a look at these features and research both PaX and Grsecurity so you can fully understand what they provide.

What Is Grsecurity?

Grsecurity is another series of patches that can be applied to the Linux kernel. These patches will allow changes to policy, provide a tool for RBAC, and add new policies. Grsecurity and PaX are built to work together and reinforce one another.

Why Are We Doing This?

Well, to get the most secure system that you can possibly get you’re going to want to start compiling your own kernel. The kernel is, for various reasons, where security belongs (more on this in future posts.) Now, on a default install of some distro like, say, Ubuntu you’re fairly secure and the tools already provided can be used to really lock the system down (think AppArmor) but if you’re looking to secure your system against sophisticated and targeted attacks PAX and Grsecurity are the way to go.

Keep in mind that the settings I use are for my own system. A server would be able to use more settings because it wouldn’t have to worry about, for example, the sound working.

You will also need to do this for new updates from kernel.org in order to get the latest security patches, among other things. Not even Grsecurity will keep an unpatched system safe.

So, since we’re done with the intro…

Setting Up Our System

Before we even download source code we need to make sure we have everything necessary to actually compile and configure our kernel. Let’s start by installing some software: (I’ll assume sudo/root for this guide.)

That’s (hopefully) everything you need to compile, package, and configure the kernel. (You may need to install gcc and plugins.)

Now let’s set up a working folder and cd to it.

# mkdir /usr/src/linux/

# cd /usr/src/linux/

That’s it for now.

Downloading The Linux Kernel And Patches

In this example I’ll be using the Linux Kernel 3.2.18 (the same steps will work for any kernel, just change the patches) and the Grsecurity patch that goes along with it. I’m assuming you’re still in /usr/src/linux/.

Once the download has completed you can extract linux-3.2.18.tar.bz2 to ./linux-3.2.18.

Patching The Kernel

This part is fairly simple, move to your new linux-3.2.18 folder and patch.

# cd ./linux-3.2.18

# patch -p1 < /usr/src/linux/grsecurity-2.9-3.2.18-201205301835.patch

This should patch the kernel without any interaction necessary.

Configuring The Kernel

(note: I am taking screenshots of a 3.3.6 kernel with my own configuration, yours may vary. Once size does not fit all – do research!)

#make menuconfig

You’ll be brought to this screen:

Note: If you want to enable KERNEXEC you must go to Processor Features and disable Xen/Paravirtualization.

Navigate (arrow keys) to Security Options and hit ‘Enter’ and…:

Configuring Grsecurity:

As you can see there are many options here. You can use preconfigured options (Low, Medium, High) or, as I suggest, Custom.

The following settings are suggestions that work for me. They can make or break systems depending on usage. A server might need completely different settings, for example. You’ll probably be compiling a few kernels just to see what breaks when.

As with Grsecurity we’re presented with quite a few options. PAX settings can very easily break things so my settings will likely vary from what your settings could be. I will note specific areas where there are known conflicts.

**As of version 12.8 the closed source ATI FGLRX drivers are compatible with the above. Earlier versions will NOT work.

Other Security Settings

Exit your PaX options and go back to the security settings.

Set ” Low address space for LSM to protect from user allocation” to 65536 or 32768:

This is the portion of low virtual memory which should be protected
from userspace allocation. Keeping a user from writing to low pages
can help reduce the impact of kernel NULL pointer bugs.

For most ia64, ppc64 and x86 users with lots of address space
a value of 65536 is reasonable and should cause no problems.
On arm and other archs it should not be higher than 32768.
Programs which use vm86 functionality or have some need to map
this low address space will need the permission specific to the
systems running LSM.

Time To Compile And Install

(Note: You may want to see the last section ‘Other Tips’ before you compile.)

# make-kpkg clean

# make-kpkg --initrd --append-to-version "grsec1.0" kernel_image

The above command might give you an error in which case you’ll need to type it out. It’s a wordpress issue where it combines the –‘s.

Compiling will take a while. Probably a half hour, potentially multiple hours depending on the hardware. When you’re done there will be a package in /usr/src/linux.

# cd /usr/src/linux

# dpkg -i ./<name.of.deb>

Reboot into the new kernel and that’s it.

It’s a lot of work to set it up but it should only take you a few minutes after the first time. Compiling is what takes the longest but it can be done in the background.

If something breaks just boot into the last kernel you used. Simple.

Other Tips

Since you’re taking the time to compile this thing you may as well get the most out of it. I suggest you go through the settings and disable any drivers not related/ enable or disable features.

You should also open /usr/src/linux/linux-3.2.18/makefile and find the line:

# Add user supplied CPPFLAGS, AFLAGS and CFLAGS as the last assignments

And have the following line as: (add the -march=native -pipe -mtune=native)

KBUILD_CPPFLAGS += $(KCPPFLAGS) -march=native -pipe -mtune=native

pipe will speed up compile time significantly for systems with lots of RAM. march=native will optimize the kernel for your specific CPU. Don’t use -march=native if you are compiling for another system with a different CPU (same for mtune).

Take the time to disable drivers that you won’t use. It’ll speed up compile time and produce a smaller kernel.

Highlights

The highlights of this new kernel are:

1) Improved ASLR

2) RBAC

3) Hardened Chroots (if you use Chromium or Google Chrome the sandbox makes use of Chroots so you’ve just made it more secure.)

4) Exploit bruteforcing is much harder

5) A generally more secure system with stricter policies and checks

6) Optimized kernel if you followed the “notes.”

This was a long post so if I’ve left anything out or misrepresented something leave a comment and let me know. RBAC is worth its own full post so I’ll get to that later (setting it up on any user-oriented OS is difficult.) For now you can see this wiki page to learn how to set RBAC up, but it’s not so simple.

It would be great if you could setup a repository and release a 3.2 series grsecurity image targeted at the desktop. The Tails project could really use it. Tails uses a program called Tor to let its users communicate anonymously. Activists and others use this who are targeted.

Sysfs/debugfs restriction breaks only pulse audio, alsa works.
Runtime read-only mount protection is not only handy for servers but also for desktop/workstations where you want the kernel to obey /etc/fstab /etc/mtab ro(read-only) settings (enforce).

I think it’s a rare situation where a user wants read-only mount protection like what Grsecurity enforces. Naturally different situations call for different security measures but typical users won’t want or need this.

Yep, it’s just a PulseAudio issue. I can specify that in the guide I suppose.

In addition if you disable RBAC in the kernel config and specify Apparmor or SELinux as default you have both grsecurity and Apparmor/SELinux protection.
Apparmor profiles are a little bit easier to set up and furthermore most mainstream apps have a Apparmor profile which one could easily download and activate.

Thanks for a great article.
You have mention that you are compiling the grsecurity for Linux 3.3.6.
I have version 3.3.
Can you please write if there are changes, I will really appreciate if you patch the changes.
Thank you
Jons

You see, Ubuntu carries quite some patches on top of mainline kernels and does so for good reasons. Their backports mostly deal with driver stability (most notably GPU drivers) but they also carry patches like AuFS and OverlayFS without which you can’t make a LiveCD.

Ubuntu desktop kernel configuration is also carried as a series of patches (see https://wiki.ubuntu.com/Kernel/FAQ/UbuntuDelta). Which means you’ll have to configure the absolutely generic kernel for yourself by hand. Definitely NOT a great idea!
And essential Ubuntu packaging is not present in mainline kernels either, so I have some doubts regarding correctness of generated .deb. I’m pretty sure it breaks dkms, for example.

This can be mitigated by using Ubuntu kernels as a base, but I’m not sure if the patches will still apply cleanly.

But either way you have to manually monitor the mainline kernel for security updates, not to mention manually re-downloading, re-patching and re-building the kernel when an update is issued.
Better mention it in the article – I’ve learned the hard way that people are extremely fond of ‘install and forget’ approach to software, even kernels, completely ignoring the possibility of security updates.

If you’re using the latest kernel you still get security patches. You lose some of the Ubuntu specific things, but none of it effects security, and your system still boots, you just loose a few features. I will note this.

True. I just wanted to point out that you have to care about security updates to a custom-built kernel by yourself, you no longer get them automatically with the rest of system updates.
I’m sure it’s obvious to you but I’ve seen so many people just installing .deb’s for mainline kernels and forgetting about them, that I feel it should be explicitly noted in the guide.

Nothing major, as far as I know. Some drivers may be missing. One example is that before Ubuntu used the 3.5 kernel they backported Seccomp Mode 2 Filter support to their stable kernel, but I don’t know anything critical that’s missing.

Grsec has at least one of the patches (which allows aa-status), but it does not have the network patch. So network rules are not enforced on mainline kernels. I’ll talk to Brad and see if he’ll add it.

Would I be correct that this can not be used at the moment due to GrSecurity patch file being out dated with the most recent kernel 3.10.6?
because 3.10.5 is in testing and well its already outdated lol.
despite their being minor differences to the kernel itself from 3.10.5 – 3.10.6.
Also anyone using the test version I.E. beta had any severe bugs with it?
at this point i feel its better to wait.
thanks for the tutorials on your site though i learn something new every once in awhile like DNS crypt hardening awesome.
Guess for now its just Apparmor.

Once in a while they fall a patch (or two) behind, but usually it’s out within a day or two. I’m currently using 3.10.5, just waiting for the next release. I don’t really mind if they go a bit behind, because even if there were a security release in those versions, it would likely take much longer to attack a Grsecurity kernel, long enough for a patch to come out.

In terms of severe bugs, the only one I can think of is that PAGE_KERNEXEC breaks ‘update-grub’, so I always keep a vanilla kernel installed, otherwise updating the kernel becomes impossible.

Hey, I followed your guide to the letter. However, I noticed two things – the first is that even though I made sure that the Kernel and patch versions were an exact match, I got “Reversed (or previously applied) patch detected! Assume -R? [n]” a huge number of times. After I got past that and entered the screen for GRsecurity, I didn’t see any of the options that you listed above.

The lineKBUILD_CPPFLAGS += $(KCPPFLAGS) -march=native -pipe -mtune=native
gave me a lot of trouble.

For some reason, -march=native triggers an error that causes the compilation under gcc version 4.8.2 (from Ubuntu) to abort with my cpu.

(The -pipe and -mtune=native arguments, separately and in combination, work fine, however.)
This is not really an issue related to grsecurity, of course, but I mention it in case there are other inexperienced readers who are following the guide here and also experiencing problems. It’s taken me a week working through different options to figure out what was causing the error. I initially thought I had made a bad choice in the grsecurity configuration, but I found that I can’t even get the kernel to compile by itself without the grsecurity patch included.

Other thought that probably will actually be more useful for others —
if you set the CONCURRENCY_LEVEL during the make-kpkg command to take advantage of multicore cpus, it can save a lot of time.
For example, while -pipe added to the Makefile shaves a mere 2 seconds off the 30 minutes it takes to compile the vanilla kernel without the grsecurity patch, adding the CONCURRENCY_LEVEL (with -j9, for an 8-core cpu, where the j value is set to be your number of cpu cores + 1) cut the compilation time by almost 70%:

The menuconfig help description for PAX_LATENT_ENTROPY reads as follows:

“By saying Y here the kernel will instrument some kernel code to extract some entropy from both original and artificially created program state. This will help especially embedded systems where there is little ‘natural’ source of entropy normally. The cost is some slowdown of the boot process and fork and irq processing. When pax_extra_latent_entropy is passed on the kernel command line, entropy will be extracted from up to the first 4GB of RAM while the runtime memory allocator is being initialized. This costs even more slowdown of the boot process. Note that the implementation requires a gcc with plugin support, i.e., gcc 4.5 or newer. You may need to install the supporting headers explicitly in addition to the normal gcc package. ***Note that entropy extracted this way is not cryptographically secure!***”

… but then there is this, from the grsecurity website:

“Providing enough randomness, especially at early boot, has been a constant struggle for Linux. While on the Intel platform, Ivy Bridge and newer processor models that support RDRAND generally eliminate the problem of entropy starvation at early boot and runtime, older processors and those of other architectures continue to be plagued by this problem. The demonstrable effects of this include weak private keys on embedded systems, weak randomness on VM guests, predictable stack canaries, and more. PaX’s latent entropy plugin included in grsecurity addresses the lack of randomness by modifying the instruction streams of functions called only during boot as well as some specific functions at runtime that perform some unpredictable computations from the perspective of an attacker. These functions are modified to produce a nondeterministic value based on the flow of execution. ***At several points in the boot process and at runtime, this pseudorandom value is mixed in to the entropy pool.*** Further, a kernel compiled with this plugin will have the various random pools in the kernel initialized at compile-time to random contents.”

… So the configuration help says that it’s not cryptographically secure, but then their website documentation says that it’s being mixed into the entropy pool. Can you explain if this going to somehow weaken security for cryptographic applications that require random data from the entropy pool?

Thanks, that clears things up, I think. So basically this entropy is only being created/used during the early boot process and then later the normal entropy generation process would be the only one used, correct?

Yeah, basically. When the system boots up more devices it can start gathering more entropy from new sources, and that can be cryptographically secure. But right at the beginning, when almost nothing is booted up, it still needs entropy but for non-cryptographic things.