Please submit only technical tips that will help other TidBITS readers better use their Macs, iPhones, iPads, and related software and hardware. All product announcements should be sent to releases@tidbits.com.

Tip title*

Your tip*

URL

Enter the URL to a Web page that supports your tip.

Linked text

Enter the name of the page linked above.

Your name*

Your email*

* indicates required fields

To help us avoid automated posts and spam, please enter the words below.

When you submit a tip, you give us permission to use it. Read our terms for more details. All submissions are reviewed before publication.

Our terms: By submitting a tip, you agree to assign TidBITS Publishing Inc., a non-exclusive, worldwide, perpetual license to reproduce, publish, and distribute your tip in connection with the TidBITS Web site and associated products in any media. You agree that you created the content you submitted, and that you have the right to assign us this license. You give us permission to use your name, but your email address won't be publicly displayed or shared. We review all submissions before publication, and reserve the right to select which submissions we feel are appropriate for our readers and to edit those we publish.

Our terms: We reserve the right to edit or delete any comment, so please post thoughtfully. We use your email address only to send you a one-time verification message confirming that you posted this comment. We also store your address to allow you to verify using other Web browsers in the future. For more info, see our privacy policy.

The Sandbox Conundrum: Security vs. Innovation

In an email message to all registered developers, Apple announced last week that it has, once again, extended the deadline for developers to comply with new security requirements in order to publish their apps on the Mac App Store — this time until 1 June 2012.

The company had originally required developers to adopt a “sandbox” model for their apps by 1 March 2012 or risk being shut out of the Mac App Store. Sandboxing enables Mac OS X to limit an application’s access to system resources like files stored outside the app’s sandbox and the network in an attempt to limit the possibility that malware will make its way onto a user’s computer (see “Lion Security: Building on the iOS Foundation,” 12 August 2011).

The deadline extension comes hard on the heels of the announcement of OS X 10.8 Mountain Lion, and reveals the extent to which Apple is struggling to bridge its desire for a highly secure environment for users with developers’ desire to have access to every feature of the operating system in order to give their software the features users want.

Pros and Cons of Sandboxing -- The basic idea behind sandboxing is simple: Apps that run in a sandboxed environment are prevented from accessing system resources that could lead to the disclosure of sensitive information (like the file system or the Clipboard) or that could be used for nefarious purposes (like the network). Developers must explicitly ask Apple for permission to access each resource and be prepared to justify their request as part of the Mac App Store submission process. Apple, in turn, may grant exceptions, called “entitlements,” that allow an app to work outside of the sandbox under limited circumstances.

In theory, the result is an environment which, as long as every app participates in sandboxing and there are no security issues in the operating system, is entirely safe for the end user. This doesn’t just mean “no malware,” but also (and perhaps more importantly, given that the vast majority of apps are written by honest developers) a computer that is impervious to crashes caused by apps stepping on each other’s toes.

This is how things work in iOS, where apps are sandboxed by default and the App Store is the only distribution channel for developers (barring jailbreaking, which Apple does not support or condone, though a few apps for jailbroken iPhones offer compelling capabilities, some of which Apple has even added to iOS after the fact). There is no denying that, apart from a few hiccups, Apple’s mobile operating system has been successful in both providing a highly reliable system and maintaining security for users.

In practical terms, however, sandboxing prevents applications from interacting with the operating system and with one another. Many popular Mac OS X apps require sweeping access to the operating system to function at all: Audio Hijack Pro, LaunchBar, FastScripts, Default Folder X, and Keyboard Maestro, to list just a few common examples. These apps do nothing harmful, and are in fact providing functionality that their users desperately want, but they are essentially incompatible with the “secure” environment that Apple is trying to implement.

Apple has been working to find a middle ground with the developer community, responding to the criticism that has surrounded sandboxing by introducing more entitlements for developers (so sandboxed apps can do more) and by twice extending the deadline by which apps must be sandboxed to be accepted into the Mac App Store.

So what’s the problem?

Follow the Money -- Unfortunately, while sandboxing is certainly a viable technical solution to the problems of security and reliability, Apple has muddied the waters by making it the price of entry into the Mac App Store.

On the one hand, this seems reasonable. Apple is offering a carrot — distribution in the Mac App Store — in exchange for developers sandboxing their apps. This isn’t the iOS world, where the App Store is the only source for software, so it would seem a relatively simple equation. Alas, it’s not nearly so cut and dried, for two reasons.

First, the sandboxing requirement for the Mac App Store is new, and many apps that have already found success in the Mac App Store cannot be sandboxed without losing significant functionality. Apple announced that such apps can remain in the Mac App Store without sandboxing after the deadline, and can even be updated with bug fixes. To be clear, Apple said:

Starting June 1, if you have an existing app on the Mac App Store that is not sandboxed, you may still submit bug fix updates without sandboxing your app. In addition, if you have technical issues that prevent you from sandboxing your app by June 1, let us know.

Left unanswered is the question of what happens when a developer with a Mac App Store app that cannot be sandboxed wants to release a feature update. Will Apple approve such an update, or simply reject it on the grounds that it doesn’t meet the sandboxing requirements?

We would argue that Apple should accept such updates, regardless of the inconsistency, for the simple reason that to reject them would break Apple’s implicit agreement with customers that updates to Mac App Store apps will be made available, for free, through the Mac App Store. Remember, when you purchase something through the Mac App Store, you’re Apple’s customer, not the developer’s, and Apple does not share any customer contact information with developers. So there’s currently no way developers could even take over support for such Mac App Store orphans, leading to a situation where everyone loses: developers lose customers, Apple and the Mac App Store take a reputation hit, and users lose access to updated software they
have purchased.

The second reason the sandboxing for Mac App Store distribution equation isn’t simple is that it won’t be long before many users — particularly those new to the platform — see the Mac App Store as the only source for Mac software and won’t buy software directly from developers. In essence, Apple will end up with nearly the same level of market control as in iOS, while still claiming that the Mac software market is open to all comers.

And that’s where we end up with a chilling effect. Established developers may have sufficiently large audiences and marketing machines to launch new software outside the Mac App Store, but for new developers, the cost and effort may outweigh the possible earnings. As soon as the Mac App Store becomes sufficiently dominant that developers feel they can’t succeed outside of it, we all suffer from the lack of software that might have been.

Where We’re Going -- Let’s be real. There’s no question that security and reliability are important, and Apple has no Orwellian plans for world domination that involve getting rid of pesky developers who won’t toe Apple’s corporate line.

Instead, it seems that Apple believes that the success of iOS is due in part to apps that use the operating system in exactly the way Apple intends them to and can never step outside their own sandboxes.

This is not just a matter of freakish control over every detail (goodness knows there is still a vast number of truly awful iOS apps, and even some of Apple’s own apps are pretty weak). When Apple engineers build developer tools with specific usage patterns in mind, they can make sure that, as long as those patterns are followed strictly, the operating system and its apps run in harmony and give users the best possible experience, or at least the most predictable one.

If, on the other hand, developers are given free rein to interact with the operating system in any way they wish, it’s entirely possible that they will stumble upon an approach that works most of the time, but that causes the operating system to become unstable on occasion.

This is nothing new — since the launch of the Mac App Store, apps have had to abide by Apple’s public API policy (which says that developers may use only Apple’s published methods of interacting with the operating system) to be allowed in. Sandboxing simply extends that concept to include techniques and methods that make apps “good citizens” by accessing system resources in a predictable fashion and not creating potential security hazards.

As a result, we think sandboxing is here to stay. Developers will continue to fume, but Apple appears to believe, as a result of its iOS experience, that using a sandboxed model aligns Mac apps better with the needs of its users. Progress will be slow as Apple tries to provide developers more entitlements, but eventually the deadline extensions will stop coming and access to the Mac App Store will only be granted to those who play by its new rules.

Viable Alternatives? -- Is there any other approach Apple could use that would provide an equivalent level of security while not stifling developer innovation and the advantages it brings to users?

The big unknown is exactly why Apple feels that sandboxing is sufficiently important to create such headaches for a large segment of the developer community. That’s a particularly interesting question in light of the upcoming Gatekeeper technology in Mountain Lion that will create three classes of applications: those that can be trusted because they are downloaded from the Mac App Store, those that are distributed outside the Mac App Store but can be trusted because they are digitally signed by their developers, and everything else (see “Gatekeeper Slams the Door on Mac Malware Epidemics,” 16 February 2012). Even without sandboxing, it would seem that Apple approval in the Mac App
Store would be sufficient to bump an app into the highest level of trust. (And if it’s not sufficient, doesn’t that imply Apple’s approval process adds little value?)

In a world where sandboxing is required for entry into the Mac App Store, Gatekeeper changes nothing for developers — non-sandboxed apps must still be distributed outside the Mac App Store regardless of whether or not they’re digitally signed, so all the business and innovation concerns remain.

But what if Apple changed the sandboxing carrot and offered sandboxed apps a fast track through the approval process? After all, since sandboxed apps are forced to be good citizens, approval would presumably require less investigation. And with the time saved by streamlining the process for sandboxed apps, Apple could spend more time investigating exactly what non-sandboxed apps are doing. Users would still benefit from the trust engendered by Apple’s approval process, and Apple could even build a revocation capability into the App Store app in the event that something bad were to slip through.

Developers currently find Apple’s approval process frustrating, unpredictable, and opaque, and they would undoubtedly do whatever was technically feasible to smooth it out. Those who couldn’t play within a sandbox would understand that the approval process would be longer and more drawn-out, but they wouldn’t be shut out of the Mac App Store solely for wanting to provide users features that aren’t possible in a sandboxed app.

One of the significant factors in this situation is that Apple’s communication with developers is weak at best. A number of developers we queried said that Apple was relatively communicative and helpful regarding questions about how to make sandboxing work. But for questions whose answers lie outside the sandbox model, developers are told to file bugs, which are then ignored, closed, or moved to “internal tracking.”

Long gone are the days when Apple had evangelists whose job it was to convince developers to adopt new Apple technologies and who in turn served as a communication channel for developers to talk back to Apple. Nowadays, nearly the only way to talk directly to Apple engineers is at the annual Worldwide Developer Conference, and even there, most of the communication travels from Apple to developers.

Perhaps if Apple were to reopen those lines of communication, it would be possible to come up with a solution that would work for everyone. In the meantime, it appears that sandboxing will make the Mac a safer and more reliable environment at the cost of stillborn innovations that would interact with the operating system and with other apps in novel ways. And that seems a shame.

Comments about The Sandbox Conundrum: Security vs. Innovation(Comments are closed.)

In your discussion of this topic, you're passing over a category of non-sandbox-friendly apps that's significant (to me, at least :))---software development tools. I'm one of the authors of DrRacket, and complying with sandboxing regulations will be essentially impossible for any full-featured IDE that allows users to develop programs.

I realize that this is currently a smaller market than many others, but as programming becomes more and more common, this family of "sandbox bad behavers" may become more significant, and it will be ever-harder for open-source developers to provide IDEs that run on OS X.

Oh yes, development tools are by definition left out of the sandbox. I think losing them from the Mac App Store would be a shame, though at least they're the sort of software that people are more likely to seek out. I don't think too many people download IDEs on impulse. :-)

The problem that I have with sandboxing is Apple's continuing insistence that all apps in the MAS be sandboxed. I think that a far better course would be to require apps that CAN be sandboxed to be sandboxed, and to speed their course through the approval process. Apps that cannot be sandboxed should still be allowed, however, just vetted more carefully by the app approval process.

I am also concerned with the apparent requirement for apps to keep documents inside their own application space.

It seems to me that Microsoft is handling this area better in the upcoming Metro-style app area of Windows 8 than is Apple is handling it in the Mac App Store.

The rule for Metro-style apps is tighter: they MUST be offered only through the Microsoft store (IT departments dealing with managed machines aside.) And they have limited access to the underlying OS.

However, there is a nice range of "contracts" available. For example, if I write an email client (highly unlikely), I offer a sharing contract that other apps can use to send me some data to email. It's then my job to send the data. The source app has no right to do email; I have no right to wander around the machine looking for data to send. If my app doesn't explicitly offer a contract, it can't go wandering around, although it can make use of some other app's contract to do something authorized.

That's just an executive summary of the true situation, and even so may be wrong in detail.

As to getting apps into the store, a large portion of the Microsoft vetting is automated, including the supported contracts. And MS provides the vetting tool to developers so they can check their work as they go. MS also provides a tool that tracks the app through the MS vetting process once it has been submitted. The annoying surprises should be minimal. (MS charges the same 30% that Apple does, but reduces that to ??% once an app has grossed $25,000.)

For the classic side of the house, only available with X86-X64 apps, sourcing is like earlier Windows (although Internet Explorer has gotten quite good at keeping bad guys out), except that I think app signing is required. (Note that for ARM devices ("WOA" or Windows on ARM), only Metro-style is allowed.)

None of this will matter unless people buy Windows 8 machines starting later this year. I think they will--I'm often wrong about such things.