Blog

OS X 10.8 Gatekeeper in Depth

As you can tell from my TidBITS review of Gatekeeper, I think this is an important advancement in consumer security. There are a lot of in-depth technical aspects that didn’t fit in that article, so here’s an additional Q&A for those of you with a security background who care about these sorts of things. I’m skipping the content from the TidBITS article, so you might want to read that first.

Will Gatekeeper really make a difference?

I think so. Right now the majority of the small population of malware we see for Macs is downloaded trojans and tools like Mac Defender that download through the browser. While there are plenty of ways to circumvent Gatekeeper, most of them are the sorts of things that will raise even uneducated users’ hackles. Gatekeeper attacks the economics of widespread malware. It conveys herd immunity. If most users use it (and as the default, that’s extremely likely) it will hammers on the profitability of phishing-based trojans.

To attackers going after individual users, Gatekeeper is barely a speed bump. But in terms of the entire malware ecosystem, it’s much more effective – more like tire-slashing spikes.

How does Gatekeeper work?

Gatekeeper is an extension of the quarantine features first implemented in Mac OS X 10.5. When you download files using certain applications a “quarantine bit” is set (more on that in a second). In OS X 10.5-10.7 when you open a file Launch Services looks for that attribute. If it’s set, it informs the user that the program was downloaded from the Internet and asks if they still want to run it.

Users click through everything, so that doesn’t accomplish much.

In 10.6 and 10.7 it also checks the file for any malware before running, using a short list that Apple now updates daily (as needed). If malware is detected it won’t let you open the file.

If the application was code signed, the file’s digital certificate is also checked and used to validate integrity. This prevents tampered applications from running.

In Mac OS X 10.8 (Mountain Lion), Gatekeeper runs all those checks and validates the source of the download. I believe this is done using digital certificates, rather than another extended attribute. If the file is from an approved source (the Mac App Store or a recognized Developer ID) then it’s allowed to run. Gatekeeper also checks developer certificates against a blacklist.

So here is the list of checks:

Is the quarantine attribute set?

Is the file from an approved source (per the user’s settings)?

Is the digital certificate on the blacklist?

Has the signed application been tampered with?

Does the application contain a known malware signature?

If it passes those checks, it can run.

What is the quarantine bit?

The quarantine bit is an extended file attribute set by certain applications on downloaded files. Launch Services checks it when running an application. When you approve an application (first launch) the attribute is removed, so you are never bothered again for that version.

This is why some application updates trigger quarantine and others don’t… the bit is set by the downloading application, not the operating system.

What applications set the quarantine bit?

Most Apple applications, like Safari, Firefox, Mail.app, and a really big list in /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/Exceptions.plist. Plus any applications where developers implement it as part of their download features.

In other words, most things a consumer will use to download files off the Internet. But the clearly they won’t catch everything, so there are still applications that can download and avoid Gatekeeper. System utilities like curl, aren’t protected.

What apps aren’t protected?

Anything already on your system is grandfathered in.

Files transferred or installed using fixed media like DVDs, USB drives, and other portable media.

Files downloaded by applications that don’t set the quarantine bit.

Scripts and other code that isn’t executable.

So will this protect me from Flash and Java malware?

Nope. Although they are somewhat sandboxed in browsers (which varies widely by browser), applets and other code run just fine in their container, and aren’t affected or protected.

Now we just need Adobe to sandbox Flash like they did on Windows.

What is the Developer ID?

This is a new digital certificate issued by Apple for code signing. It is integrated into XCode. Any developer in the Mac App Developer Program can obtain one for free.

Apple does not review apps signed with a Developer ID, but if they find a developer doing things they shouldn’t they can revoke that certificate.

These are signed by an Apple subroot that is separate from the Mac App Store subroot.

How are Developer ID certificates revoked?

Mountain Lion includes a blacklist that Apple updates every 24 hours.

If a malicious application is found and Apple revokes the certificate, will it still run?

Yes, if it has already run once and had the quarantine bit cleared. Apple does not remove the app from your system, although they said they can use Software Update to clean any widespread malware as they did with Mac Defender.

What about a malicious application in the Mac App Store?

Apple will remove the application from the app store. This does not remove it from your system, and it would also need to be cleaned with a software update.

If we start seeing a lot of this kind of problems, I expect this mechanism to change.

Does this mean all Mac applications require code signing?

No, but code signing is required for all App Store and Developer ID applications.

Starting in Lion, Apple includes extensive support for code signing and sandboxing. Developers can break out and sign different components of their applications and implement pretty robust sandboxing. While I expect most developers to stick with basic signing, the tools are there for building some pretty robust applications (as they are on Windows – Microsoft is pretty solid here as well, although few developers take advantage of it).

What role does sandboxing play?

All Mac App Store applications must implement sandboxing by March 1st, long before Mountain Lion is released. Sandbox entitlements are pretty restrictive right now, although there are ways to circumvent some of them for malicious developers and compromised applications.

Between review by Apple and sandboxing, I expect to see far fewer malicious apps appear in the Mac App Store than other places. And even vulnerable apps will be harder to do exploit.

Some developers of major popular applications can’t put their apps in the Mac App Store due to sandboxing. Apple is actively working on expanding entitlements to expand the number of apps that can distribute through the store. For example, they added a temporary entitlement to allow Apple Events to a specific target application, and entitlements to arbitrary directories and files as long as you specify an exact location.

Sandboxing is definitely fodder for a future article. I even have an outline!

What setting should I use?

I’m going with Mac App Store and known developers for myself, but I will set most family members to Mac App Store only.

Does Mountain Lion have any other security enhancements?

Yep: full ASLR (Address Space Layout Randomization) for the kernel

So are Macs immune to malware?

Nope. Not at all. But with Mountain Lion we gain full ASLR down to the kernel level and additional anti-exploitation protection. Gatekeeper should both protect individual users and impede the spread of Mac trojans. Lion already has DEP (Data Execution Prevention), easy-to-use developer support for strong encryption, and Find My Mac in case you need to remotely wipe a lost or stolen Mac (assuming it’s encrypted and online). Combine that with the Mac App Store and Apple’s move to mandatory sandboxing, and we have not only a reasonably secure platform, but a reasonably safe place to get most of our applications.

I think Microsoft is still ahead on some of their OS enhancements, but differences between the Mac and Windows ecosystems, combined with improvements in Mountain Lion, will give Macs a serious advantage. I hope Microsoft is smart about their app store, which could level the playing field again.

Is Apple taking security more seriously?

In the past, despite being a Mac fan, I’ve been pretty critical of how Apple handles a lot of security. While I still don’t agree with how they handle everything, I’ve noticed a massive change in the past 2 years. With Lion, Apple for the first time invited certain security researchers to evaluate pre-release software (albeit under NDA) without forcing them to pay for a Developer Program subscription. With Mountain Lion they pre-briefed an outside security type for the first time ever. They have hired a bunch of very smart and experienced security experts.

I believe Apple recognizes they aren’t the kid in the corner hanging out with all the artsy types any more. They know that the bottom line will be affected if users no longer feel safe on their products. So they are taking security much more seriously.

But this is still Apple. The culture of secrecy is definitely intact, and don’t expect them to talk about vulnerabilities and exploits like Microsoft. Apple talks when Apple wants to, and they will patch and update on their own schedule, based on their own priorities. They have hardened the platform, and now we need to watch and see how they respond to future security incidents.

I will keep this article updated as more questions come in, but right now I’m having a hard time thing of anything I missed.

Contact

About

Securosis is an information security research and advisory firm dedicated to transparency, objectivity, and quality. We are totally obsessed with improving the practice of information security. Our job is to save you money and help you do your job better and faster by helping you cut through the noise and providing clear, actionable, pragmatic advice on securing your organization.