Canonical Voices

What Jamie Strandboge talks about

This may be totally obvious to many but I thought I would point out a problem I had with slow boot and how I solved it to highlight a neat feature of systemd to investigate the problem.

I noticed that I was having terrible boot times, so I used the `systemd-analyze critical-chain` command to see what the problem was. This showed me that `network-online.target` was taking 2 minutes. Looking at the logs (and confirming with `systemd-analyze blame`), I found it was timing out because `systemd-networkd` failed to bring interfaces either online or to fail (I was in an area where I had not connected to the existing wifi before and the wireless interface was scanning instead of failing). I looked around and found that I had no configuration for systemd-networkd (/etc/systemd/network was empty) or for netplan (/etc/netplan was empty), so I simply ran `sudo systemctl disable systemd-networkd` (leaving NetworkManager enabled) and I then had a very fast boot.

I need to file a bug on the cause of the problem, but I found the `systemd-analyze` command so helpful, I wanted to share. :)

Most of this has been discussed on mailing lists, blog entries, etc, while developing Ubuntu Touch, but I wanted to write up something that ties together these conversations for Snappy. This will provide background for the conversations surrounding hardware access for snaps that will be happening soon on the snappy-devel mailing list.

Background

Ubuntu Touch has several goals that all apply to Snappy:

we want system-image upgrades

we want to replace the distro archive model with an app store model for Snappy systems

we want developers to be able to get their apps to users quickly

we want a dependable application lifecycle

we want the system to be easy to understand and to develop on

we want the system to be secure

we want an app trust model where users are in control and express that control in tasteful, easy to understand ways

Snappy adds a few things to the above (that pertain to this conversation):

we want the system to be bulletproof (transactional updates with rollbacks)

we want the system to be easy to use for system builders

we want the system to be easy to use and understand for admins

Let’s look at what all these mean more closely.

system-image upgrades

we want system-image upgrades

we want the system to be bulletproof (transactional updates with rollbacks)

We want system-image upgrades so updates are fast, reliable and so people (users, admins, snappy developers, system builders, etc) always know what they have and can depend on it being there. In addition, if an upgrade goes bad, we want a mechanism to be able to rollback the system to a known good state. In order to achieve this, apps need to work within the system and live in their own area and not modify the system in unpredictable ways. The Snappy FHS is designed for this and the security policy enforces that apps follow it. This protects us from malware, sure, but at least as importantly, it protects us from programming errors and well-intentioned clever people who might accidentally break the Snappy promise.

app store

we want to replace the distro archive model with an app store model

we want developers to be able to get their apps to users quickly

Ubuntu is a fantastic distribution and we have a wonderfully rich archive of software that is refreshed on a cadence. However, the traditional distro model has a number of drawbacks and arguably the most important one is that software developers have an extremely high barrier to overcome to get their software into users hands on their own time-frame. The app store model greatly helps developers and users desiring new software because it gives developers the freedom and ability to get their software out there quickly and easily, which is why Ubuntu Touch is doing this now.

In order to enable developers in the Ubuntu app store, we’ve developed a system where a developer can upload software and have it available to users in seconds with no human review, intervention or snags. We also want users to be able to trust what’s in Ubuntu’s store, so we’ve created store policies that understand the Ubuntu snappy system such that apps do not require any manual review so long as the developer follows the rules. However, the Ubuntu Core system itself is completely flexible– people can install apps that are tightly confined, loosely confined, unconfined, whatever (more on this, below). In this manner, people can develop snaps for their own needs and distribute them however they want.

It is the Ubuntu store policy that dictates what is in the store. The existing store policy is in place to improve the situation and is based on our experiences with the traditional distro model and attempts to build something app store-like experiences on top of it (eg, MyApps).

application lifecycle

dependable application lifecycle

This has not been discussed as much with Snappy for Ubuntu Core, but Touch needs to have a good application lifecycle model such that apps cannot run unconstrained and unpredictably in the background. In other words, we want to avoid problems with battery drain and slow systems on Touch. I think we’ve done a good job so far on Touch, and this story is continuing to evolve.

(I mention application lifecycle in this conversation for completeness and because application lifecycle and security work together via the app’s application id)

security

we want the system to be secure

we want an app trust model where users are in control and express that control in tasteful, easy to understand ways

Everyone wants a system that they trust and that is secure, and security is one of the core tenants of Snappy systems. For Ubuntu Touch, we’ve created a
system that is secure, that is easy to use and understand by users, and that still honors relevant, meaningful Linux traditions. For Snappy, we’ll be adding several additional security features (eg, seccomp, controlled abstract socket communication, firewalling, etc).

Our security story and app store policies give us something that is between Apple and Google. We have a strong security story that has a number of similarities to Apple, but a lightweight store policy akin to Google Play. In addition to that, our trust model is that apps not needing manual review are untrusted by the OS and have limited access to the system. On Touch we use tasteful, contextual prompting so the user may trust the apps to do things beyond what the OS allows on its own (simple example, app needs access to location, user is prompted at the time of use if the app can access it, user answers and the decision is remembered next time).

Snappy for Ubuntu Core is different not only because the UI supports a CLI, but also because we’ve defined a Snappy for Ubuntu Core user that is able to run the ‘snappy’ command as someone who is an admin, a system builder, a developer and/or someone otherwise knowledgeable enough to make a more informed trust decision. (This will come up again later, below)

easy to use

we want the system to be easy to understand and to develop on

we want the system to be easy to use for system builders

we want the system to be easy to use and understand for admins

We want a system that is easy to use and understand. It is key that developers are able to develop on it, system builders able to get their work done and admins can install and use the apps from the store.

For Ubuntu Touch, we’ve made a system that is easy to understand and to develop on with a simple declarative permissions model. We’ll refine that for Snappy and make it easy to develop on too. Remember, the security policy is there not just so we can be ‘super secure’ but because it is what gives us the assurances needed for system upgrades, a safe app store and an altogether bulletproof system.

As mentioned, the system we have designed is super flexible. Specifically, the underlying system supports:

(Keep in mind the Ubuntu App Store policy will auto-accept apps falling under ‘1’ and trigger manual review for the others)

The above all works today (though it isn’t always friendly– we’re working on that) and the developer is in control. As such, Snappy developers have a plethora of options and can create snaps with security policy for their needs. When the developer wants to ship the app and make it available to all Snappy users via the Ubuntu App Store, then the developer may choose to work within the system to have automated reviews or choose not to and manage the process via manual reviews/commercial relationship with Canonical.

Moving forward

The above works really well for Ubuntu Touch, but today there is too much friction with regard to hardware access. We will make this experience better without compromising on any of our goals. How do we put this all together, today, so people can get stuff done with snappy without sacrificing on our goals, making it harder on ourselves in the future or otherwise opening Pandora’s box? We don’t want to relax our security policy, because we can’t make the bulletproof assurances we are striving for and it would be hard to tighten the security. We could also add some temporary security policy that adds only certain accesses (eg, serial devices) but, while useful, this is too inflexible. We also don’t want to have apps declare the accesses themselves to automatically adds the necessary security policy, because this (potentially) privileged access is then hidden from the Snappy for Ubuntu Core user.

The answer is simple when we remember that the Snappy for Ubuntu Core user (ie, the one who is able to run the snappy command) is knowledgeable enough to make the trust decision for giving an app access to hardware. In other words, let the admin/developer/system builder be in control.

immediate term

The first thing we are going to do is unblock people and adjust snappy to give the snappy core user the ability to add specific device access to snap-specific security policy. In essence you’ll install a snap, then run a command to give the snap access to a particular device, then you’re done. This simple feature will unblock developers and snappy users immediately while still supporting our trust-model and goals fully. Plus it will be worth implementing since we will likely always want to support this for maximum flexibility and portability (since people can use traditional Linux APIs).

The user experience for this will be discussed and refined on the mailing list in the coming days.

short term

After that, we’ll build on this and explore ways to make the developer and user experience better through integration with the OEM part and ways of interacting with the underlying system so that the user doesn’t have to necessarily know the device name to add, but can instead be given smart choices (this can have tie-ins to the web interface for snappy too). We’ll want to be thinking about hotpluggable devices as well.

Since this all builds on the concept of the immediate term solution, it also supports our trust-model and goals fully and is relatively easy to implement.

future

Once we have the above in place, we should have a reasonable experience for snaps needing traditional device access. This will give us time to evaluate how people are accessing hardware and see if we can make things even better by using frameworks and/or a hardware abstraction layer. In this manner, snaps can program to an easy to use API and the system can mediate access to the underlying hardware via that API.

Ubuntu Core with Snappy was recently announced and a key ingredient for snappy is security. Snappy applications are confined by AppArmor and the confinement story for snappy is an evolution of the security model for Ubuntu Touch. The basic concepts for confined applications and the AppStore model pertain to snappy applications as well. In short, snappy applications are confined using AppArmor by default and this is achieved through an easy to understand, use and developer-friendly system. Read the snappy security specification for all the nitty gritty details.

Last time I discussed AppArmor, I talked about new features in Ubuntu 13.10 and a bit about ApplicationConfinement for Ubuntu Touch. With the release of Ubuntu 14.04 LTS, several improvements were made:

Mediation of signals

Mediation of ptrace

Various policy updates for 14.04, including new tunables, better support for XDG user directories, and Unity7 abstractions

Signal and ptrace mediation

Prior to Ubuntu 14.04 LTS, a confined process could send signals to other processes (subject to DAC) and ptrace other processes (subject to DAC and YAMA). AppArmor on 14.04 LTS adds mediation of both signals and ptrace which brings important security improvements for all AppArmor confined applications, such as those in the Ubuntu AppStore and qemu/kvm machines as managed by libvirt and OpenStack.

When developing policy for signal and ptrace rules, it is important to remember that AppArmor does a cross check such that AppArmor verifies that:

the process sending the signal/performing the ptrace is allowed to send the signal to/ptrace the target process

the target process receiving the signal/being ptraced is allowed to receive the signal from/be ptraced by the sender process

This demonstrates that the ‘cat’ binary running under the ‘foo’ profile was unable to read the contents of a /proc entry (in my test, /proc/11300/environ). To allow this process to read /proc entries for unconfined processes, the following rule can be used:

ptrace (read) peer=unconfined,

If the receiving process was confined, the log entry would say ‘peer=”<profile name>”‘ and you would adjust the ‘peer=unconfined’ in the rule to match that in the log denial. In this case, because unconfined processes implicitly can be readby all other processes, we don’t need to specify the cross check rule. If the target process was confined, the profile for the target process would need a rule like this:

This shows that ‘bash’ running under the ‘foo’ profile tried to send the ‘term’ signal to an unconfined process (in my test, I used ‘kill 11300’) and was blocked. Signal rules use ‘read’ and ‘send to determine access, so we can add a rule like so to allow sending of the signal:

signal (send) set=("term") peer=unconfined,

Like with ptrace, a cross-check is performed with signal rules but implicit rules allow unconfined processes to send and receive signals. If pid 11300 were confined, you would adjust the ‘peer=’ in the rule of the foo profile to match the denial in the log, and then adjust the target profile to have something like:

signal (receive) set=("term") peer=foo,

Signal and ptrace rules are very flexible and the AppArmor base abstraction in Ubuntu 14.04 LTS has several rules to help make profiling and transitioning to the new mediation easier:

# Checking for PID existence is quite common so add it by default for now
signal (receive, send) set=("exists"),

Note the above uses the new ‘@{profile_name}’ AppArmor variable, which is particularly handy with ptrace and signal rules. See man 5 apparmor.d for more details and examples.

14.10

Work still remains and some of the things we’d like to do for 14.10 include:

Finishing mediation for non-networking forms of IPC (eg, abstract sockets). This will be done in time for the phone release.

Have services integrate with AppArmor and the upcoming trust-store to become trusted helpers (also for phone release)

Continue work on netowrking IPC (for 15.04)

Continue to work with the upstream kernel on kdbus

Work continued on LXC stacking and we hope to have stacked profiles within the current namespace for 14.10. Full support for stacked profiles where different host and container policy for the same binary at the same time should be ready by 15.04

Various fixes to the python userspace tools for remaining bugs. These will also be backported to 14.04 LTS

Integration with several services as part of the ApplicationConfinement work in support of click packages and the Ubuntu appstore

Better support for policy generation via the aa-easyprof tool and apparmor-easyprof-ubuntu policy

Native AppArmor support in Upstart

DBus mediation

Prior to Ubuntu 13.10, access to the DBus system bus was on/off and there was no mediation of the session bus or any other DBus buses, such as the accessibility bus. 13.10 introduces fine-grained DBus mediation. In a nutshell, you define ‘dbus’ rules in your AppArmor policy just like any other rules. When an application that is confined by AppArmor uses DBus, the dbus-daemon queries the kernel on if the application is allowed to perform this action. If it is, DBus proceeds normally, if not, DBus denies the access and logs it to syslog. An example denial is:

This rule says that the application is allowed to use the ‘Hello’ method on the ‘org.freedesktop.DBus’ interface of the ‘/org/freedesktop/DBus’ object for the process bound to the ‘org.freedesktop.DBus’ name on the ‘session’ bus. That is fine-grained indeed!

However, rules don’t have to be that fine-grained. For example, all of the following are valid rules:

Because dbus-daemon is the one performing the mediation, DBus denials are logged to syslog and not kern.log. Recent versions of Ubuntu log kernel messages to /var/log/syslog, so I’ve gotten in the habit of just looking there for everything

The message content of DBus traffic is not examined

The userspace tools don’t understand DBus rules yet. That means aa-genprof, aa-logprof and aa-notify don’t work with these new rules. The userspace tools are being rewritten and support will be added in a future release.

The less fine-grained the rule, the more access is permitted. So ‘dbus,’ allows unrestricted access to DBus.

Responses to messages are implicitly allowed, so if you allow an application to send a message to a service, the service is allowed to respond without needing a corresponding rule.

dbus-daemon is considered a trusted helper (it integrates with AppArmor to enforce the mediation) and is not confined by default.

As a transitional step, existing policy for packages in the Ubuntu archive that use DBus will continue to have full access to DBus, but future Ubuntu releases may provide fine-grained DBus rules for this software. See ‘man 5 apparmor.d’ for more information on DBus mediation and AppArmor.

Application confinement

Ubuntu will support an app store model where software that has not gone through the traditional Ubuntu archive process is made available to users. While this greatly expands the quantity of quality software available to Ubuntu users, it also introduces new security risks. An important part of addressing these risks is to run applications under confinement. In this manner, apps are isolated from each other and are limited in what they can do on the system. AppArmor is at the heart of the Ubuntu ApplicationConfinement story and is already working on Ubuntu 13.10 for phones in the appstore. A nice introduction for developers on what the Ubuntu trust model is and how apps work within it can be found at http://developer.ubuntu.com.

In essence, a developer will design software with the Ubuntu SDK, then declare what type of application it is (which determines the AppArmor template to use), then declares any addition policy groups that the app needs. The templates and policy groups define AppArmor file, network, DBus and anything other rules that are needed. The software is packaged as a lightwight click package and when it is installed, an AppArmor click hook is run which creates a versioned profile for the application based on the templates and policy groups. On Unity 8, application lifecycle makes sure that the app is launched under confinement via an upstart job. For other desktop environments, a desktop file is generated in ~/.local/share/applications that prepends ‘aa-exec-click’ to the Exec line. The upstart job and ‘aa-exec-click’ not only launch the app under confinement, but also setup the environment (eg, set TMPDIR to an application specific directory). Various APIs have been implemented so apps can access files (eg, Pictures via the gallery app), connect to services (eg, location and online accounts) and work within Unity (eg, the HUD) safely and in a controlled and isolated manner.

14.04

Work on APIs and AppArmor integration of services to work better on the converged device (ie, with traditional desktop applications)

Work with the upstream kernel on kdbus so we are ready for when that is available

Finish the LXC stacking work to allow different host and container policy for the same binary at the same time

While Mir already handles keyboard and mouse sniffing, we’d like to integrate with Mir in other ways where applicable (note, X mediation for keyboard/mouse sniffing, clipboard, screen grabs, drag and drop, and xsettings is not currently scheduled nor is wayland support. Both are things we’d like to have though, so if you’d like to help out here, join us on #apparmor on OFTC to discuss how to contribute)

Integration with several services as part of the ApplicationConfinement work in support of click packages and the Ubuntu appstore

Better support for policy generation via the aa-easyprof tool and apparmor-easyprof-ubuntu policy

Native AppArmor support in Upstart

DBus mediation

Prior to Ubuntu 13.10, access to the DBus system bus was on/off and there was no mediation of the session bus or any other DBus buses, such as the accessibility bus. 13.10 introduces fine-grained DBus mediation. In a nutshell, you define ‘dbus’ rules in your AppArmor policy just like any other rules. When an application that is confined by AppArmor uses DBus, the dbus-daemon queries the kernel on if the application is allowed to perform this action. If it is, DBus proceeds normally, if not, DBus denies the access and logs it to syslog. An example denial is:

This rule says that the application is allowed to use the ‘Hello’ method on the ‘org.freedesktop.DBus’ interface of the ‘/org/freedesktop/DBus’ object for the process bound to the ‘org.freedesktop.DBus’ name on the ‘session’ bus. That is fine-grained indeed!

However, rules don’t have to be that fine-grained. For example, all of the following are valid rules:

Because dbus-daemon is the one performing the mediation, DBus denials are logged to syslog and not kern.log. Recent versions of Ubuntu log kernel messages to /var/log/syslog, so I’ve gotten in the habit of just looking there for everything

The message content of DBus traffic is not examined

The userspace tools don’t understand DBus rules yet. That means aa-genprof, aa-logprof and aa-notify don’t work with these new rules. The userspace tools are being rewritten and support will be added in a future release.

The less fine-grained the rule, the more access is permitted. So ‘dbus,’ allows unrestricted access to DBus.

Responses to messages are implicitly allowed, so if you allow an application to send a message to a service, the service is allowed to respond without needing a corresponding rule.

dbus-daemon is considered a trusted helper (it integrates with AppArmor to enforce the mediation) and is not confined by default.

As a transitional step, existing policy for packages in the Ubuntu archive that use DBus will continue to have full access to DBus, but future Ubuntu releases may provide fine-grained DBus rules for this software. See ‘man 5 apparmor.d’ for more information on DBus mediation and AppArmor.

Application confinement

Ubuntu will support an app store model where software that has not gone through the traditional Ubuntu archive process is made available to users. While this greatly expands the quantity of quality software available to Ubuntu users, it also introduces new security risks. An important part of addressing these risks is to run applications under confinement. In this manner, apps are isolated from each other and are limited in what they can do on the system. AppArmor is at the heart of the Ubuntu ApplicationConfinement story and is already working on Ubuntu 13.10 for phones in the appstore. A nice introduction for developers on what the Ubuntu trust model is and how apps work within it can be found at http://developer.ubuntu.com.

In essence, a developer will design software with the Ubuntu SDK, then declare what type of application it is (which determines the AppArmor template to use), then declares any addition policy groups that the app needs. The templates and policy groups define AppArmor file, network, DBus and anything other rules that are needed. The software is packaged as a lightwight click package and when it is installed, an AppArmor click hook is run which creates a versioned profile for the application based on the templates and policy groups. On Unity 8, application lifecycle makes sure that the app is launched under confinement via an upstart job. For other desktop environments, a desktop file is generated in ~/.local/share/applications that prepends ‘aa-exec-click’ to the Exec line. The upstart job and ‘aa-exec-click’ not only launch the app under confinement, but also setup the environment (eg, set TMPDIR to an application specific directory). Various APIs have been implemented so apps can access files (eg, Pictures via the gallery app), connect to services (eg, location and online accounts) and work within Unity (eg, the HUD) safely and in a controlled and isolated manner.

14.04

Work on APIs and AppArmor integration of services to work better on the converged device (ie, with traditional desktop applications)

Work with the upstream kernel on kdbus so we are ready for when that is available

Finish the LXC stacking work to allow different host and container policy for the same binary at the same time

While Mir already handles keyboard and mouse sniffing, we’d like to integrate with Mir in other ways where applicable (note, X mediation for keyboard/mouse sniffing, clipboard, screen grabs, drag and drop, and xsettings is not currently scheduled nor is wayland support. Both are things we’d like to have though, so if you’d like to help out here, join us on #apparmor on OFTC to discuss how to contribute)

I’ve been involved with Ubuntu for quite a while now, and part of my journey has included accumulating a bunch of Ubuntu T-shirts. I now have so many that I can’t possibly wear them all, so my wife gave me a really cool gift today: a quilt made out of a bunch of my old Ubuntu T-shirts! :) Check it out:

I’ve been involved with Ubuntu for quite a while now, and part of my journey has included accumulating a bunch of Ubuntu T-shirts. I now have so many that I can’t possibly wear them all, so my wife gave me a really cool gift today: a quilt made out of a bunch of my old Ubuntu T-shirts! :) Check it out:

Last time I discussed AppArmor, gave some background and some information on how it can be used. This time I’d like to discuss how AppArmor is used within Ubuntu. The information in this part applies to Ubuntu 12.10 and later, and unless otherwise noted, Ubuntu 12.04 LTS (and because we also push our changes into Debian, much of this will apply to Debian as well).

/etc/apparmor.d

Ubuntu follows the upstream policy layout and naming conventions and uses the abstractions extensively. A few abstractions are Ubuntu-specific and they are prefixed with ‘ubuntu’. Eg /etc/apparmor.d/abstractions/ubuntu-browsers.

Ubuntu encourages the use of the include files in /etc/apparmor.d/local in any shipped profiles. This allows administrators to make profile additions and apply overrides without having to change the shipped profile (will need to reload the profile with apparmor_parser, see /etc/apparmor.d/local/README for more information)

All profiles in Ubuntu use ‘#include <tunables/global>’, which pulls in a number of tunables: ‘@{PROC}’ from ‘tunables/proc’, ‘@{HOME}’ and ‘@{HOMEDIRS}’ from ‘tunables/home’ and @{multiarch} from ‘tunables/multiarch’

In addition to ‘tunables/home’, Ubuntu utilizes the ‘tunables/home.d/ubuntu’ file so that ‘@{HOMEDIRS}’ is preseedable at installation time, or adjustable via ‘sudo dpkg-reconfigure apparmor’

Binary caches in /etc/apparmor.d/cache are used to speed up boot time.

Ubuntu uses the /etc/apparmor.d/disable and /etc/apparmor.d/force-complain directories. Touching (or symlinking) a file with the same name as the profile in one of these directories will cause AppArmor to either skip policy load (eg disable/usr.sbin.rsyslogd) or load in complain mode (force-complain/usr.bin.firefox)

Boot

Policy load happens in 2 stages during boot:

within the job of a package with an Upstart job file

via the SysV initscript (/etc/init.d/apparmor)

For packages with an Upstart job and an AppArmor profile (eg, cups), the job file must load the AppArmor policy prior to execution of the confined binary. As a convenience, the /lib/init/apparmor-profile-load helper is provided to simplify Upstart integration. For packages that ship policy but do not have a job file (eg, evince), policy must be loaded sometime before application launch, which is why stage 2 is needed. Stage 2 will (re)load all policy. Binary caches are used in both stages unless it is determined that policy must be recompiled (eg, booting a new kernel).

dhclient is a corner case because it needs to have its policy loaded very early in the boot process (ie, before any interfaces are brought up). To accommodate this, the /etc/init/network-interface-security.conf upstart job file is used.

Packaging

AppArmor profiles are shipped as Debian conffiles (ie, the package manager treats them specially during upgrades). AppArmor profiles in Ubuntu should use an include directive to include a file from /etc/apparmor.d/local so that local site changes can be made without modifying the shipped profile. When a package ships an AppArmor profile, it is added to /etc/apparmor.d then individually loaded into the kernel (with caching enabled). On package removal, any symlinks/files in /etc/apparmor.d/disable, /etc/apparmor.d/force-complain and /etc/apparmor.d are cleaned up. Developers can use dh_apparmor to aid in shipping profiles with their packages.

Profiles and applications

Ubuntu ships a number of AppArmor profiles. The philosophy behind AppArmor profiles on Ubuntu is that the profile should add a meaningful security benefit while at the same time not introduce regressions in default or common functionality. Because it is all too easy for a security mechanism to be turned off completely in order to get work done, Ubuntu won’t ship overly restrictive profiles. If we can provide a meaningful security benefit with an AppArmor profile in the default install while still maintaining functionality for the vast majority of users, we may ship an enforcing profile. Unfortunately, because applications are not designed to run under confinement or are designed to do many things, it can be difficult to confine these applications while still maintaining usability. Sometimes we will ship disabled-by-default profiles so that people can opt into them if desired. Usually profiles are shipped in the package that provides the confined binary (eg, tcpdump ships its own AppArmor profile). Some in progress profiles are also offered in the apparmor-profiles package and are in complain mode by default. When filing AppArmor bugs in Ubuntu, it is best to file the bug against the application that ships the profile.

In addition to shipped profiles, some applications have AppArmor integration built in or have AppArmor confinement applied in a non-standard way.

Libvirt

An AppArmor sVirt driver is provided and enabled by default for libvirt managed QEMU virtual machines. This provides strong guest isolation and userspace host protection for virtual machines. AppArmor profiles are dynamically generated in /etc/apparmor.d/libvirt and usually you won’t have to worry about AppArmor confinement. If needed, profiles for the individual machines can be adjusted in /etc/apparmor.d/libvirt/libvirt-<uuid> or in all virtual machines in /etc/apparmor.d/abstractions/libvirt-qemu. See /usr/share/doc/libvirt-bin/README.Debian.gz for details.

LXC

LXC in Ubuntu uses AppArmor to help make sure files in the container cannot access security-sensitive files on the host. lxc-start is confined by its own restricted profile which allows mounting in the container’s tree and, just before executing the container’s init, transitioning to the container’s own profile. See LXC in precise and beyond for details. Note, currently the libvirt-lxc sVirt driver does not have AppArmor support (confusingly, libvirt-lxc is a different project than LXC, but there are longer term plans to integrate the two and/or add AppArmor support to libvirt-lxc).

Firefox

Ubuntu ships a disabled-by-default profile for Firefox. While it is known to work well in the default Ubuntu installation, Firefox, like all browsers, is a very complex piece of software that can do much more than simply surf web pages so enabling the profile in the default install could affect the overall usability of Firefox in Ubuntu. The goals of the profile are to provide a good usability experience with strong additional protection. The profile allows for the use of plugins and extensions, various helper applications, and access to files in the user’s HOME directory, removable media and network filesystems. The profile prevents execution of arbitrary code, malware, reading and writing to sensitive files such as ssh and gpg keys, and writing to files in the user’s default PATH. It also prevents reading of system and kernel files. All of this provides a level of protection exceeding that of normal UNIX permissions. Additionally, the profile’s use of includes allows for great flexibility for tightly confining firefox. /etc/apparmor.d/usr.bin.firefox is a very restricted profile, but includes both /etc/apparmor.d/local/usr.bin.firefox and /etc/apparmor.d/abstractions/ubuntu-browsers.d/firefox. /etc/apparmor.d/abstractions/ubuntu-browsers.d/firefox contains other include files for tasks such as multimedia, productivity, etc and the file can be manipulated via the aa-update-browser command to add or remove functionality as needed.

Chromium

A complain-mode profile for chromium-browser is available in the apparmor-profiles package. It uses the same methodology as the Firefox profile (see above) with a strict base profile including /etc/apparmor.d/abstractions/ubuntu-browsers.d/chromium-browser whichcan then include any number of additional abstractions (also configurable via aa-update-browser).

Lightdm guest session

The guest session in Ubuntu is protected via AppArmor. When selecting the guest session, Lightdm will transition to a restrictive profile that disallows access to others’ files.

libapache2-mod-apparmor

The libapache2-mod-apparmor package ships a disabled-by-default profile in /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2. This profile provides a permissive profile for Apache itself but allows the administrator to add confinement policy for individual web applications as desired via AppArmor’s change_hat() mechanism (note, apache2-mpm-prefork must be used) and an example profile for phpsysinfo is provided in the apparmor-profiles package. For more information on how to use AppArmor with Apache, please see /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2 as well as the upstream documentation.

PAM

AppArmor also has a PAM module which allows great flexibility in setting up policies for different users. The idea behind pam_apparmor is simple: when someone uses a confined binary (such as login, su, sshd, etc), that binary will transition to an AppArmor role via PAM. Eg, if ‘su’ is configured for use with pam_apparmor, when a user invokes ‘su’, PAM is consulted and when the PAM session is started, pam_apparmor will change_hat() to a hat that matches the username, the primary group, or the DEFAULT hat (configurable). These hats (typically) provide rudimentary policy which declares the transition to a role profile when the user’s shell is started. The upstream documentation has an example of how to put this all together for ‘su’, unconfined admin users, a tightly confined user, and somewhat confined default users.

aa-notify

aa-notify is a very simple program that can alert you when there are denials on the system. aa-notify will report any new AppArmor denials by consulting /var/log/kern.log (or /var/log/audit/audit.log if auditd is installed). For desktop users who install apparmor-notify, aa-notify is started on session start via /etc/X11/Xsession.d/90apparmor-notify and will watch the logs for any new denials and report them via notify-osd. Server users can add something like ‘/usr/bin/aa-notify -s 1 -v’ to their shell startup files (eg, ~/.profile) to show any AppArmor denials within the last day whenever they login. See ‘man 8 aa-notify’ for details.

Current limitations

For all AppArmor can currently do and all the places it is used in Ubuntu, there are limitations in AppArmor 2.8 and lower (ie, what is in Ubuntu and other distributions). Right now, it is great for servers, network daemons and tools/utilities that process untrusted input. While it can provide a security benefit to client applications, there are currently a number of gaps in this area:

Access to the DBus system bus is on/off and there is no mediation of the session bus or any other buses that rely on Unix abstract sockets, such as the accessibility bus

AppArmor does not provide environment filtering beyond having the ability to clear the very limited set of glibc secure-exec variables

Display management mediation is not present (eg, it doesn’t protect against X snooping)

Networking mediation is too coarse-grained (eg you can allow ‘tcp’ but cannot restrict the binding port, integrate with a firewall or utilize secmark)

LXC/containers support is functional, but not complete (eg, it doesn’t allow different host and container policy for the same binary at the same time)

AppArmor doesn’t currently integrate with other client technologies that might be useful (eg gnome-keyring, signon/gnome-online-accounts and gsettings) and there is no facility to dynamically update a profile via a user prompt like a file/open dialog

Next time I’ll discuss ongoing and future work to address these limitations.

Last time I discussed AppArmor, gave some background and some information on how it can be used. This time I’d like to discuss how AppArmor is used within Ubuntu. The information in this part applies to Ubuntu 12.10 and later, and unless otherwise noted, Ubuntu 12.04 LTS (and because we also push our changes into Debian, much of this will apply to Debian as well).

/etc/apparmor.d

Ubuntu follows the upstream policy layout and naming conventions and uses the abstractions extensively. A few abstractions are Ubuntu-specific and they are prefixed with ‘ubuntu’. Eg /etc/apparmor.d/abstractions/ubuntu-browsers.

Ubuntu encourages the use of the include files in /etc/apparmor.d/local in any shipped profiles. This allows administrators to make profile additions and apply overrides without having to change the shipped profile (will need to reload the profile with apparmor_parser, see /etc/apparmor.d/local/README for more information)

All profiles in Ubuntu use ‘#include <tunables/global>’, which pulls in a number of tunables: ‘@{PROC}’ from ‘tunables/proc’, ‘@{HOME}’ and ‘@{HOMEDIRS}’ from ‘tunables/home’ and @{multiarch} from ‘tunables/multiarch’

In addition to ‘tunables/home’, Ubuntu utilizes the ‘tunables/home.d/ubuntu’ file so that ‘@{HOMEDIRS}’ is preseedable at installation time, or adjustable via ‘sudo dpkg-reconfigure apparmor’

Binary caches in /etc/apparmor.d/cache are used to speed up boot time.

Ubuntu uses the /etc/apparmor.d/disable and /etc/apparmor.d/force-complain directories. Touching (or symlinking) a file with the same name as the profile in one of these directories will cause AppArmor to either skip policy load (eg disable/usr.sbin.rsyslogd) or load in complain mode (force-complain/usr.bin.firefox)

Boot

Policy load happens in 2 stages during boot:

within the job of a package with an Upstart job file

via the SysV initscript (/etc/init.d/apparmor)

For packages with an Upstart job and an AppArmor profile (eg, cups), the job file must load the AppArmor policy prior to execution of the confined binary. As a convenience, the /lib/init/apparmor-profile-load helper is provided to simplify Upstart integration. For packages that ship policy but do not have a job file (eg, evince), policy must be loaded sometime before application launch, which is why stage 2 is needed. Stage 2 will (re)load all policy. Binary caches are used in both stages unless it is determined that policy must be recompiled (eg, booting a new kernel).

dhclient is a corner case because it needs to have its policy loaded very early in the boot process (ie, before any interfaces are brought up). To accommodate this, the /etc/init/network-interface-security.conf upstart job file is used.

Packaging

AppArmor profiles are shipped as Debian conffiles (ie, the package manager treats them specially during upgrades). AppArmor profiles in Ubuntu should use an include directive to include a file from /etc/apparmor.d/local so that local site changes can be made without modifying the shipped profile. When a package ships an AppArmor profile, it is added to /etc/apparmor.d then individually loaded into the kernel (with caching enabled). On package removal, any symlinks/files in /etc/apparmor.d/disable, /etc/apparmor.d/force-complain and /etc/apparmor.d are cleaned up. Developers can use dh_apparmor to aid in shipping profiles with their packages.

Profiles and applications

Ubuntu ships a number of AppArmor profiles. The philosophy behind AppArmor profiles on Ubuntu is that the profile should add a meaningful security benefit while at the same time not introduce regressions in default or common functionality. Because it is all too easy for a security mechanism to be turned off completely in order to get work done, Ubuntu won’t ship overly restrictive profiles. If we can provide a meaningful security benefit with an AppArmor profile in the default install while still maintaining functionality for the vast majority of users, we may ship an enforcing profile. Unfortunately, because applications are not designed to run under confinement or are designed to do many things, it can be difficult to confine these applications while still maintaining usability. Sometimes we will ship disabled-by-default profiles so that people can opt into them if desired. Usually profiles are shipped in the package that provides the confined binary (eg, tcpdump ships its own AppArmor profile). Some in progress profiles are also offered in the apparmor-profiles package and are in complain mode by default. When filing AppArmor bugs in Ubuntu, it is best to file the bug against the application that ships the profile.

In addition to shipped profiles, some applications have AppArmor integration built in or have AppArmor confinement applied in a non-standard way.

Libvirt

An AppArmor sVirt driver is provided and enabled by default for libvirt managed QEMU virtual machines. This provides strong guest isolation and userspace host protection for virtual machines. AppArmor profiles are dynamically generated in /etc/apparmor.d/libvirt and usually you won’t have to worry about AppArmor confinement. If needed, profiles for the individual machines can be adjusted in /etc/apparmor.d/libvirt/libvirt-<uuid> or in all virtual machines in /etc/apparmor.d/abstractions/libvirt-qemu. See /usr/share/doc/libvirt-bin/README.Debian.gz for details.

LXC

LXC in Ubuntu uses AppArmor to help make sure files in the container cannot access security-sensitive files on the host. lxc-start is confined by its own restricted profile which allows mounting in the container’s tree and, just before executing the container’s init, transitioning to the container’s own profile. See LXC in precise and beyond for details. Note, currently the libvirt-lxc sVirt driver does not have AppArmor support (confusingly, libvirt-lxc is a different project than LXC, but there are longer term plans to integrate the two and/or add AppArmor support to libvirt-lxc).

Firefox

Ubuntu ships a disabled-by-default profile for Firefox. While it is known to work well in the default Ubuntu installation, Firefox, like all browsers, is a very complex piece of software that can do much more than simply surf web pages so enabling the profile in the default install could affect the overall usability of Firefox in Ubuntu. The goals of the profile are to provide a good usability experience with strong additional protection. The profile allows for the use of plugins and extensions, various helper applications, and access to files in the user’s HOME directory, removable media and network filesystems. The profile prevents execution of arbitrary code, malware, reading and writing to sensitive files such as ssh and gpg keys, and writing to files in the user’s default PATH. It also prevents reading of system and kernel files. All of this provides a level of protection exceeding that of normal UNIX permissions. Additionally, the profile’s use of includes allows for great flexibility for tightly confining firefox. /etc/apparmor.d/usr.bin.firefox is a very restricted profile, but includes both /etc/apparmor.d/local/usr.bin.firefox and /etc/apparmor.d/abstractions/ubuntu-browsers.d/firefox. /etc/apparmor.d/abstractions/ubuntu-browsers.d/firefox contains other include files for tasks such as multimedia, productivity, etc and the file can be manipulated via the aa-update-browser command to add or remove functionality as needed.

Chromium

A complain-mode profile for chromium-browser is available in the apparmor-profiles package. It uses the same methodology as the Firefox profile (see above) with a strict base profile including /etc/apparmor.d/abstractions/ubuntu-browsers.d/chromium-browser whichcan then include any number of additional abstractions (also configurable via aa-update-browser).

Lightdm guest session

The guest session in Ubuntu is protected via AppArmor. When selecting the guest session, Lightdm will transition to a restrictive profile that disallows access to others’ files.

libapache2-mod-apparmor

The libapache2-mod-apparmor package ships a disabled-by-default profile in /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2. This profile provides a permissive profile for Apache itself but allows the administrator to add confinement policy for individual web applications as desired via AppArmor’s change_hat() mechanism (note, apache2-mpm-prefork must be used) and an example profile for phpsysinfo is provided in the apparmor-profiles package. For more information on how to use AppArmor with Apache, please see /etc/apparmor.d/usr.lib.apache2.mpm-prefork.apache2 as well as the upstream documentation.

PAM

AppArmor also has a PAM module which allows great flexibility in setting up policies for different users. The idea behind pam_apparmor is simple: when someone uses a confined binary (such as login, su, sshd, etc), that binary will transition to an AppArmor role via PAM. Eg, if ‘su’ is configured for use with pam_apparmor, when a user invokes ‘su’, PAM is consulted and when the PAM session is started, pam_apparmor will change_hat() to a hat that matches the username, the primary group, or the DEFAULT hat (configurable). These hats (typically) provide rudimentary policy which declares the transition to a role profile when the user’s shell is started. The upstream documentation has an example of how to put this all together for ‘su’, unconfined admin users, a tightly confined user, and somewhat confined default users.

aa-notify

aa-notify is a very simple program that can alert you when there are denials on the system. aa-notify will report any new AppArmor denials by consulting /var/log/kern.log (or /var/log/audit/audit.log if auditd is installed). For desktop users who install apparmor-notify, aa-notify is started on session start via /etc/X11/Xsession.d/90apparmor-notify and will watch the logs for any new denials and report them via notify-osd. Server users can add something like ‘/usr/bin/aa-notify -s 1 -v’ to their shell startup files (eg, ~/.profile) to show any AppArmor denials within the last day whenever they login. See ‘man 8 aa-notify’ for details.

Current limitations

For all AppArmor can currently do and all the places it is used in Ubuntu, there are limitations in AppArmor 2.8 and lower (ie, what is in Ubuntu and other distributions). Right now, it is great for servers, network daemons and tools/utilities that process untrusted input. While it can provide a security benefit to client applications, there are currently a number of gaps in this area:

Access to the DBus system bus is on/off and there is no mediation of the session bus or any other buses that rely on Unix abstract sockets, such as the accessibility bus

AppArmor does not provide environment filtering beyond having the ability to clear the very limited set of glibc secure-exec variables

Display management mediation is not present (eg, it doesn’t protect against X snooping)

Networking mediation is too coarse-grained (eg you can allow ‘tcp’ but cannot restrict the binding port, integrate with a firewall or utilize secmark)

LXC/containers support is functional, but not complete (eg, it doesn’t allow different host and container policy for the same binary at the same time)

AppArmor doesn’t currently integrate with other client technologies that might be useful (eg gnome-keyring, signon/gnome-online-accounts and gsettings) and there is no facility to dynamically update a profile via a user prompt like a file/open dialog

Next time I’ll discuss ongoing and future work to address these limitations.

Recently I wanted to mirror traffic from one interface to another. There is a cool utility from Martin Roesch (of Snort fame) called daemonlogger. It’s use is pretty simple (note I created the ‘daemonlogger user and group on my Ubuntu system since I wanted it to drop privileges after starting):$ sudo /usr/bin/daemonlogger -u daemonlogger \
-g daemonlogger -i eth0 -o eth1

Sure enough, examing tcpcump output, anything coming in on eth0 was mirrored to eth1. I then used a crossover cable and used tcpdump on the other system and it all worked as advertised. This could be very useful for an IDS.

Then I thought how cool it would be to do this for multiple interfaces, but then I quickly realized that would be confusing for whatever was trying to interpret that traffic. I read that someone used a combination of daemonlogger and vtun for a remote snort monitor to work in EC2, which got me playing….

In my test environment, I again used a crossover cable to connect two systems (let’s call them the ‘server’ and the ‘monitor’). The server system runs vtund. The monitor system then connects as a vtun client. When this happens, tap devices are created on the server and the monitor. The vtund server is setup to start daemonlogger on connect to mirror the traffic from eth0 to the tap0 device. The monitor system can then listen on the tap device and get all the traffic. I then created a second vtun tunnel and mirrored from eth1 to tap1. The monitor could monitor on both tap devices. Neat!

With the above, I was able to start suricata (a snort-compatible rewrite that is multithreaded) to listen to both tap0 and tap1 on the monitoring system and see all the traffic. Very cool. :)

Now, this isn’t optimized for anything– sending 2 interfaces’ worth of traffic through one interface is likely going to result in droppoed packets to the monitor if the links are busy. Could maybe use compression to help there. Also, the server really needs to have an extra interface for vtun traffic that isn’t mirrored, otherwise you end up mirroring the vtun traffic itself (in my test environment, eth2 on the server used 10.0.2.1 and the montior used 10.0.2.2). Finally, vtun doesn’t offer any meaningful line encryption so if you are doing this for anything resembling production, you’re going to want to use a dedicated network segment between the server and the monitor (a crossover cable worked fine in my test environment).

Recently I wanted to mirror traffic from one interface to another. There is a cool utility from Martin Roesch (of Snort fame) called daemonlogger. It’s use is pretty simple (note I created the ‘daemonlogger user and group on my Ubuntu system since I wanted it to drop privileges after starting):$ sudo /usr/bin/daemonlogger -u daemonlogger \
-g daemonlogger -i eth0 -o eth1

Sure enough, examing tcpcump output, anything coming in on eth0 was mirrored to eth1. I then used a crossover cable and used tcpdump on the other system and it all worked as advertised. This could be very useful for an IDS.

Then I thought how cool it would be to do this for multiple interfaces, but then I quickly realized that would be confusing for whatever was trying to interpret that traffic. I read that someone used a combination of daemonlogger and vtun for a remote snort monitor to work in EC2, which got me playing….

In my test environment, I again used a crossover cable to connect two systems (let’s call them the ‘server’ and the ‘monitor’). The server system runs vtund. The monitor system then connects as a vtun client. When this happens, tap devices are created on the server and the monitor. The vtund server is setup to start daemonlogger on connect to mirror the traffic from eth0 to the tap0 device. The monitor system can then listen on the tap device and get all the traffic. I then created a second vtun tunnel and mirrored from eth1 to tap1. The monitor could monitor on both tap devices. Neat!

With the above, I was able to start suricata (a snort-compatible rewrite that is multithreaded) to listen to both tap0 and tap1 on the monitoring system and see all the traffic. Very cool. :)

Now, this isn’t optimized for anything– sending 2 interfaces’ worth of traffic through one interface is likely going to result in droppoed packets to the monitor if the links are busy. Could maybe use compression to help there. Also, the server really needs to have an extra interface for vtun traffic that isn’t mirrored, otherwise you end up mirroring the vtun traffic itself (in my test environment, eth2 on the server used 10.0.2.1 and the montior used 10.0.2.2). Finally, vtun doesn’t offer any meaningful line encryption so if you are doing this for anything resembling production, you’re going to want to use a dedicated network segment between the server and the monitor (a crossover cable worked fine in my test environment).

A lot of exciting work has been going on with AppArmor and this multipart series will discuss where AppArmor is now, how it is currently used in Ubuntu and how it fits into the larger application isolation story moving forward.

Brief History and Background

AppArmor is a Mandatory Access Control (MAC) system which is a Linux Security Module (LSM) to confine programs to a limited set of resources. AppArmor’s security model is to bind access control attributes to programs rather than to users. AppArmor confinement is provided via profiles loaded into the kernel. AppArmor profiles can be in one of two modes: enforcement and complain. Profiles loaded in enforcement mode will result in enforcement of the policy defined in the profile as well as reporting policy violation attempts (either via syslog or auditd) such that what is not allowed in policy is denied. Profiles in complain mode will not enforce policy but instead report policy violation attempts. AppArmor is typically deployed on systems as a targeted policy where only some (eg high risk) applications have an AppArmor profile defined, but it also supports system wide policy.

Some defining characteristics of AppArmor are that it:

is root strong

is path-based

allows for mixing of enforcement and complain mode profiles

uses include files to ease development

is very lightweight in terms of resources

is easy to learn

is relatively easy to audit

AppArmor is an established technology first seen in Immunix and later integrated into SUSE and Ubuntu and their derivatives. AppArmor is also available in Debian, Mandriva, Arch, PLD, Pardus and others. Core AppArmor functionality is in the mainline Linux kernel starting with 2.6.36. AppArmor maintenance and development is ongoing.

An example AppArmor profile

Probably the easiest way to describe what AppArmor does and how it works is to look at an example, in this case the profile for tcpdump on Ubuntu 12.04 LTS:

This profile is representative of traditional AppArmor profiling for a program that processes untrusted input over the network. As can be seen:

profiles are simple text files

comments are supported in the profile

absolute paths as well as file globbing can be used when specifying file access

various access controls for files are present. From the profile we see ‘r’ (read), ‘w’ (write), ‘m’ (memory map as executable), ‘k’ (file locking), ‘l’ (creation hard links), and ‘ix’ to execute another program with the new program inheriting policy. Other access rules also exists such as ‘Px’ (execute under another profile, after cleaning the environment), ‘Cx’ (execute under a child profile, after cleaning the environment), and ‘Ux’ (execute unconfined, after cleaning the environment)

access controls for capabilities are present

access controls for networking are present

explicit deny rules are supported, to override other allow rules (eg access to @{HOME}/bin/bad.sh is denied with auditing due to ‘audit deny @{HOME}/bin/** mrwkl,’ even though general access to @{HOME} is permitted with ‘@{HOME}/** rw,’)

variables can be defined and manipulated outside the profile (#include <tunables/global> for @{PROC} and @{HOME})

AppArmor profiles are fairly easy to read and audit

Complete information on the profile language can be found in ‘man 5 apparmor.d’ as well as the AppArmor wiki.

Updating and creating profiles

AppArmor uses the directory heirarchy as described in policy layout, but most of the time, you are either updating an existing profile or creating a new one and so the files you most care about are in /etc/apparmor.d.

The AppArmor wiki has a lot of information on debugging and updating existing profiles. AppArmor denials are logged to /var/log/kern.log (or /var/log/audit/audit.log if auditd is installed). If an application is misbehaving and you think it is because of AppArmor, check the logs first. If there is an AppArmor denial, adjust the policy in /etc/apparmor.d, then reload the policy and restart the program like so:

Oftentimes it is enough to just reload the policy without unloading/loading the profile or restarting the application:

$ sudo apparmor_parser -r /etc/apparmor.d/usr.bin.foo

Creating a profile can be done either with tools or by hand. Due to the current pace and focus of development, the tools are somewhat behind and lack some features. It is generally recommended that you profile by hand instead.

When profiling, keep in mind that:

AppArmor provides an additional permission check to traditional Discretionary Access Controls (DAC). DAC is always checked in addition to the AppArmor permission checks. As such, AppArmor cannot override DAC to provide more access than what would be normally allowed.

AppArmor normalizes path names. It resolves symlinks and considers each hard link as a different access path.

AppArmor evaluates file access by pathname rather than using on disk labeling. This eases profiling since AppArmor handles all the labelling behind the scenes.

Deny rules are evaluated after allow rules and cannot be overridden by an allow rule.

Creation of files requires the create permission (implied by w) on the path to be created. Separate rules for writing to the directory of where the file resides are not required. Deletion works like creation but requires the delete permission (implied by w). Copy requires ‘r’ of the source with create and write at the destination (implied by w). Move is like copy, but also requires delete at source.

The profile must be loaded before an application starts for the confinement to take effect. You will want to make sure that you load policy during boot before any confined daemons or applications.

The kernel will rate limit AppArmor denials which can cause problems while profiling. You can avoid this be installing auditd or by adjusting rate limiting in the kernel:

$ sudo sysctl -w kernel.printk_ratelimit=0

Resources

There is a lot of documentation on AppArmor (though some is still in progress):

A lot of exciting work has been going on with AppArmor and this multipart series will discuss where AppArmor is now, how it is currently used in Ubuntu and how it fits into the larger application isolation story moving forward.

Brief History and Background

AppArmor is a Mandatory Access Control (MAC) system which is a Linux Security Module (LSM) to confine programs to a limited set of resources. AppArmor’s security model is to bind access control attributes to programs rather than to users. AppArmor confinement is provided via profiles loaded into the kernel. AppArmor profiles can be in one of two modes: enforcement and complain. Profiles loaded in enforcement mode will result in enforcement of the policy defined in the profile as well as reporting policy violation attempts (either via syslog or auditd) such that what is not allowed in policy is denied. Profiles in complain mode will not enforce policy but instead report policy violation attempts. AppArmor is typically deployed on systems as a targeted policy where only some (eg high risk) applications have an AppArmor profile defined, but it also supports system wide policy.

Some defining characteristics of AppArmor are that it:

is root strong

is path-based

allows for mixing of enforcement and complain mode profiles

uses include files to ease development

is very lightweight in terms of resources

is easy to learn

is relatively easy to audit

AppArmor is an established technology first seen in Immunix and later integrated into SUSE and Ubuntu and their derivatives. AppArmor is also available in Debian, Mandriva, Arch, PLD, Pardus and others. Core AppArmor functionality is in the mainline Linux kernel starting with 2.6.36. AppArmor maintenance and development is ongoing.

An example AppArmor profile

Probably the easiest way to describe what AppArmor does and how it works is to look at an example, in this case the profile for tcpdump on Ubuntu 12.04 LTS:

This profile is representative of traditional AppArmor profiling for a program that processes untrusted input over the network. As can be seen:

profiles are simple text files

comments are supported in the profile

absolute paths as well as file globbing can be used when specifying file access

various access controls for files are present. From the profile we see ‘r’ (read), ‘w’ (write), ‘m’ (memory map as executable), ‘k’ (file locking), ‘l’ (creation hard links), and ‘ix’ to execute another program with the new program inheriting policy. Other access rules also exists such as ‘Px’ (execute under another profile, after cleaning the environment), ‘Cx’ (execute under a child profile, after cleaning the environment), and ‘Ux’ (execute unconfined, after cleaning the environment)

access controls for capabilities are present

access controls for networking are present

explicit deny rules are supported, to override other allow rules (eg access to @{HOME}/bin/bad.sh is denied with auditing due to ‘audit deny @{HOME}/bin/** mrwkl,’ even though general access to @{HOME} is permitted with ‘@{HOME}/** rw,’)

variables can be defined and manipulated outside the profile (#include <tunables/global> for @{PROC} and @{HOME})

AppArmor profiles are fairly easy to read and audit

Complete information on the profile language can be found in ‘man 5 apparmor.d’ as well as the AppArmor wiki.

Updating and creating profiles

AppArmor uses the directory heirarchy as described in policy layout, but most of the time, you are either updating an existing profile or creating a new one and so the files you most care about are in /etc/apparmor.d.

The AppArmor wiki has a lot of information on debugging and updating existing profiles. AppArmor denials are logged to /var/log/kern.log (or /var/log/audit/audit.log if auditd is installed). If an application is misbehaving and you think it is because of AppArmor, check the logs first. If there is an AppArmor denial, adjust the policy in /etc/apparmor.d, then reload the policy and restart the program like so:

Oftentimes it is enough to just reload the policy without unloading/loading the profile or restarting the application:

$ sudo apparmor_parser -r /etc/apparmor.d/usr.bin.foo

Creating a profile can be done either with tools or by hand. Due to the current pace and focus of development, the tools are somewhat behind and lack some features. It is generally recommended that you profile by hand instead.

When profiling, keep in mind that:

AppArmor provides an additional permission check to traditional Discretionary Access Controls (DAC). DAC is always checked in addition to the AppArmor permission checks. As such, AppArmor cannot override DAC to provide more access than what would be normally allowed.

AppArmor normalizes path names. It resolves symlinks and considers each hard link as a different access path.

AppArmor evaluates file access by pathname rather than using on disk labeling. This eases profiling since AppArmor handles all the labelling behind the scenes.

Deny rules are evaluated after allow rules and cannot be overridden by an allow rule.

Creation of files requires the create permission (implied by w) on the path to be created. Separate rules for writing to the directory of where the file resides are not required. Deletion works like creation but requires the delete permission (implied by w). Copy requires ‘r’ of the source with create and write at the destination (implied by w). Move is like copy, but also requires delete at source.

The profile must be loaded before an application starts for the confinement to take effect. You will want to make sure that you load policy during boot before any confined daemons or applications.

The kernel will rate limit AppArmor denials which can cause problems while profiling. You can avoid this be installing auditd or by adjusting rate limiting in the kernel:

$ sudo sysctl -w kernel.printk_ratelimit=0

Resources

There is a lot of documentation on AppArmor (though some is still in progress):

While this may be old news to some, I only just now figured out how to conveniently use multiple identities in OpenSSH. I have several ssh keys, but only two that I want to use with the agent: one for personal use and one for work. I’d like to be able to not specify which identity to use on the command line most of the time and just use ssh like so:

$ ssh <personal>
$ ssh <work>
$ ssh -i ~/.ssh/other.id_rsa <other>

where the first two use the agent with ~/.ssh/id_rsa and ~/.ssh/work.id_rsa respectively, and the last does not use the agent. ‘man ssh_config’ tells me that the agent looks at all the different IdentityFile configuration directives (in order) and also the IdentitiesOnly option. Therefore, I can set up my ~/.ssh/config to have something like:# This makes it so that only my default identity
# and my work identity are used by the agent
IdentitiesOnly yes
IdentityFile ~/.ssh/id_rsa
IdentityFile ~/.ssh/work.id_rsa

# Default to using my work key with work domains
Host *.work1.com *.work2.com
IdentityFile ~/.ssh/work.id_rsa

While this may be old news to some, I only just now figured out how to conveniently use multiple identities in OpenSSH. I have several ssh keys, but only two that I want to use with the agent: one for personal use and one for work. I’d like to be able to not specify which identity to use on the command line most of the time and just use ssh like so:

$ ssh <personal>
$ ssh <work>
$ ssh -i ~/.ssh/other.id_rsa <other>

where the first two use the agent with ~/.ssh/id_rsa and ~/.ssh/work.id_rsa respectively, and the last does not use the agent. ‘man ssh_config’ tells me that the agent looks at all the different IdentityFile configuration directives (in order) and also the IdentitiesOnly option. Therefore, I can set up my ~/.ssh/config to have something like:# This makes it so that only my default identity
# and my work identity are used by the agent
IdentitiesOnly yes
IdentityFile ~/.ssh/id_rsa
IdentityFile ~/.ssh/work.id_rsa

# Default to using my work key with work domains
Host *.work1.com *.work2.com
IdentityFile ~/.ssh/work.id_rsa

I just learned to my chagrin that wordpress.com would update my atom feed every time I clicked the ‘Update’ button in my blog. As a result, I have recently been spamming Planet Ubuntu pretty heavily lately, for which I apologize. Thanks to jcastro and especially nigelb for figuring out that the fix was simply to have the planet use the rss feed instead of the atom feed. While this post/apology could be argued to be yet more spam, I thought I’d pass this information along so others might check their feeds and adjust accordingly. The wiki has been updated for this change, and you can reference the change made to my planet-ubuntu settings in Launchpad if you want to check your settings.