Mac Operationshttp://macops.ca
Thu, 26 Mar 2015 19:54:24 +0000en-UShourly1Security Updates leaving mach_kernel visiblehttp://macops.ca/security-updates-leaving-mach_kernel-visible/
http://macops.ca/security-updates-leaving-mach_kernel-visible/#commentsWed, 11 Mar 2015 14:33:42 +0000http://macops.ca/?p=908In the past, there have been cases where system updates for 10.8.5 (and possibly earlier versions) leave the OS X kernel (at /mach_kernel) visible to users in the Finder. This file has since moved to /System/Library/Kernels/kernel in OS X Yosemite, but previously to Yosemite it is located at /, and included in the package payload for system updates like OS X Combo/Delta and Security Updates.

OS X installers and updaters typically keep this file hidden in the Finder using a tool called SetFile, which is able to set miscellaneous file flags including the “hidden” flag. The Security Update 2015-002 for Mavericks, released on March 9, 2015, does not include any of the postinstall “actions” (miscellaneous scripts and tools executed by a master script) in the installer that were present in the 2015-001 update.

We have few admin users at my organization, but it has happened at least once that a curious admin user has wondered what this “mach_kernel” file is and moved it to the trash, only to find that their system volume will no longer boot.

Why does Apple not simply set the hidden flag in the file in the package payload, rather than depend on setting it according to a script? It is possible to set these flags on the file in a payload and not require any scripting to set a hidden attribute on a file.

We can fix this easily by distributing a script to clients that would do something like this:

While Apple’s acknowledged this issue given their knowledge base article, I still felt it’s worth opening a bug for.

]]>http://macops.ca/security-updates-leaving-mach_kernel-visible/feed/1darwinup, Apple’s Darwin Update utilityhttp://macops.ca/darwinup-apples-darwin-update-utility/
http://macops.ca/darwinup-apples-darwin-update-utility/#commentsWed, 21 Jan 2015 19:46:53 +0000http://macops.ca/?p=880Yesterday in ##osx-server, Pepijn Bruienne mentioned having stumbled upon an OS X system binary he’d never seen before, which was new to me as well: darwinup. This tool is used (or was used – public development of it seems to have stopped around OS X 10.7) for the purpose of managing versions of OS X system components by installing “roots” distributed in a variety of ways. It abstracts several different archive formats and wraps tools like curl, tar, rsync to perform its tasks.

It can install and remove packages installed via rsync-able locations and HTTP(S) URLs, and keeps track (in an SQLite database) of its activity and overwritten files such that it can roll back installations of system components to previous versions. I’ve seen it included on OS X systems as far back as OS X 10.7 (Lion) up through 10.10 (Yosemite). My immediate reaction was that this was like a basic package manager that’s included with every copy of OS X.

Digging a bit further, this tool is part of the DarwinBuild project, whose public development seems to have stopped around 10.6/10.7 (like most other macosforge projects). According to their notes, it is definitely not a package manager, however. These notes contain a much more thorough explanation of the tool than its manpage, so I’d encourage you to read through it if you’re interested in why the tool exists. The manpage has a few useful examples, such as installing components from Apple’s (similarly-abandoned) Roots repository. This repo of compiled OS X components was also completely news to me.

The darwinup tool obviously exists for testing and development purposes, so I would highly not recommend installing Apple’s old roots onto a system you care about, because they are now so outdated and could overwrite critical system components with incompatible versions. Of course, you can always roll back..

Here’s an example of installing compiled bootp tools from 10.7.2. You can also add additional -v options to print out more details about exactly what it’s doing with network and files-on-disk activity.

Again, use this with care. We can see that these bootp tools installed system components in addition to executable binaries (from a 10.7 system onto a 10.9 system), so this is just a demonstration of the capabilities of darwinup. Don’t do this at home!

From recent discussions in ##osx-server, some of us have determined that OS X’s “system data files and security updates” will only install automatically if a client is already configured to automatically check for updates. Many sysadmins managing OS X clients tend to disable this setting so that they can control the distribution of these updates, but aren’t aware that their clients are now no longer receiving Apple’s background updates for at several of its built-in security mechanisms, including XProtect and Gatekeeper.

Rich Trouton beat me to this post with his post yesterday, but it prompted me to do a bit more digging into trying to reproduce an issue that comes up when attempting the most obvious workarounds for this issue, which I’ll outline after giving some more context.

Update: Greg Neagle has come up with a simple but flexible workaround for the issue described below, which he’s implemented in Reposado and documented here. If you use Reposado (and you really should), look into the new --remove-config-data option that can be applied selectively to SUS updates you’re mirroring.

There are a couple of reasons admins usually disable automatic checks for software updates. Historically one reason was that their users weren’t administrators, and therefore couldn’t install software updates themselves even if we wanted them to. Since OS X Mavericks, by default any user can install software updates via the Mac App Store interface or just using the softwareupdate command-line tool (although ther are supported ways to configure this setting). A more important reason is that admins often want to maintain some control over when certain updates are actually rolled out to their clients, and do limited testing of system updates. This can be done using Reposado or Apple’s Software Update service included in OS X Server, both of which allow local mirroring of Apple’s own Software Update catalogs for your managed clients (Reposado does a much better job of this).

Disabling automatic checks for updates also has the effect of preventing the system from prompting the user about the new updates. This is usually done in tandem with an implementation of a client management platform like Munki, which is able to provide the user with an interface to install the system updates coming from your own server. We are replacing Apple’s user-facing mechanisms for system updates with our own.

Disabling the automatic checks is typically done by running the softwareupdate --schedule off command as part of a setup script, or setting AutomaticCheckEnabled to false in com.apple.SoftwareUpdate. We end up with an App Store preference pane that looks like this, with nothing checked:

In recent versions of OS X, Apple began using its Software Update service (which also drives system software updates that show in the App Store or via the softwareupdate command-line tool) as a mechanism for installing “background and critical” updates that are installed silently in the background with no notifications to the user. Here are several families of updates have been seen so far using this mechanism (and there are more):

We’ll take the first two as examples: XProtect stores its data in /System/Library/CoreServices/CoreTypes/XProtect* files, and Gatekeeper Configuration Data in /private/var/db/gkopaque.bundle. Both of these sets of files include standard Info.plist files with nice, always-incrementing integer version strings.

Users will never see these updates in the App Store UI. These updates may be run when other updates take place, but they also run on their own schedule.

If you run Reposado or Software Update Service in OS X Server, you’ll see these updates listed alongside standard user-facing updates. If you look at the actual .dist that go alongside these updates, you’ll notice these updates include a config-data “type” attribute up at the top in the options element. (Printing out an update’s distribution file is easy with Reposado: repoutil --dist).

You may be enabling these updates along with other updates, thinking that they will get installed. They might, but only if the clients pointing to your server have automatic checks enabled.

App Store preferences with no automatic downloads/installs, just checks.

See some of the undocumented softwareupdate Greg Neagle has documented here, namely the --background and --background-critical options. You might think, what if we just run these commands ourselves on a schedule? With these options, Software Update will schedule a scan (returning immediately) for installing only the config-data updates, but it will not actually install them if background checks are disabled.

I’d encourage you to test this for yourself: find a test client that has been been configured with background checks disabled for some time (these updates, particularly Gatekeeper, are frequently updated by Apple, often at least every couple of weeks). If you can’t find one, you can manually adjust the CFBundleShortVersionString in /private/var/db/gkopaque.bundle/Contents/Info.plist to something lower than the current version, which is listed as the update version in the Software Update catalog (again with Reposado, visible with repoutil --updates). At the time of writing this post, the current version is 52, released December 10, 2014.

That was pretty quick. Now go back and enable background checks (via the App Store Preference Pane, softwareupdate --schedule on or a sudo defaults write /Library/Preferences/com.apple.SoftwareUpdate AutomaticCheckEnabled -bool true) and re-run the sudo softwareupdate --background-critical command. The trigger will occur and softwareupdate will return immediately, but you should now see some more interesting activity in your install.log. Here’s a snippet of mine run today:

There were actually five different configuration data updates found in this run: Apple Displays metadata, Chinese Wordlist, Incompatible Kexts, XProtect and Gatekeeper Opaque Bundle. You’ll see later in the install log that they all get installed.

So given that we can ensure these are installed by enabling automatic checks, what’s to stop us just disabling automatic installations on clients, and leaving the checks enabled? Earlier I mentioned that enabling automatic checks has the effect of prompting the user to install them when recommended updates are found, and since these updates are now part of the App Store application, this prompt may also include available updates for App Store apps (which you may not want if you are already distributing using an institutional Apple ID or have disabled updates of App Store apps by regular users).

Since I wanted to ensure I could reproduce this reliably on Mavericks and Yosemite, I ended up recording a short video. This demonstrates the App Store prompt that seems to occur if automatic checks are enabled, a manual run of softwareupdate --background-critical is done when there are other available updates.

A couple of additional points. If you try to reproduce this, you may have inconsistent results because of the notification system having its own schedule of when it decides to notify the user (relative to when it did last, what feedback the user gave, etc.). The second point is that I’ve had some success on Yosemite in simply disabling the automatic check immediately after scheduling the check. In other words:

In my limited testing on Mavericks, this triggered the prompt anyway, and in my limited testing on Yosemite, it didn’t. But either way, this would be a fragile mechanism to rely on. I would not be confident that I have covered enough scenarios in testing to implement some kind of scheduled script that would run these commands in an attempt to automate these updates on clients.

An another approach to getting these updates out to clients would be via scripting or something like AutoPkg recipes to fetch the packages from Apple’s Software Update servers. This would allow an admin to deploy the packages in any way he/she sees fit. The problem with this approach, however, is that one would need to keep a close eye on exactly what conditions these updates install. This is determined by the pkg .dist files, and specifically a pile of difficult-to-read JavaScript functions doing mostly boolean logic in order to isolate updates to clients meeting specific conditions. The Gatekeeper and XProtect examples I’ve described in this article follow somewhat predictable patterns, but it’s common for updates like these to be split across multiple OSes, merged together, split again, get reposted, etc. It would be a lot of work for someone to continue to audit these distribution files to ensure that the right updates are going to the right clients. A misplaced code signature patch going into your clients’ /System/Library could have serious implications.

The fact that it’s common practice for admins to disable software update checks, and that this disables all installation of config-data updates, seems to clash with Apple’s desire to keep their OS updated quickly and transparently with configuration data that helps the systems function more reliably and securely. I consider this issue to be a security bug. My bug report on this issue is #18939764, which has been classified by Apple as an enhancement request.

]]>http://macops.ca/os-x-admins-your-clients-are-not-getting-background-security-updates/feed/11Keeping your OS X VM guest tools current with Munkihttp://macops.ca/keeping-your-os-x-vm-guest-tools-current-with-munki/
http://macops.ca/keeping-your-os-x-vm-guest-tools-current-with-munki/#commentsWed, 19 Nov 2014 16:36:31 +0000http://macops.ca/?p=712I use VMware Fusion to test client software, deployment workflows, and using virtual machines allows me to frequently take and roll back snapshots. Over time, the VMware guest OS tools tend to drift out of date with the version of Fusion, and are reported to need updates/reinstalling. Sometimes when this happens, things like pasteboard synchronization, automatic window resolution resizing and drag-and-drop file transfers stop working. I’d like to not have to manually click “Update VMware tools..” and go install the tools manually every time I notice the tools are out of date between snapshots (which on my system seems to be frequently).

Luckily, I use Munki to manage OS X clients, and it’s great at updating software. In this post I’ll walk through the few steps I did to have all my test machines configured to automatically keep their VMware tools up to date. The same logic should apply for other software management platforms like Casper, Absolute Manage or Puppet, using their respective mechanisms for customizable discoverable attributes. This technique should work for users of Parallels, if they use a sane OS X installer for their tools. VirtualBox has yet to ship with any OS X guest tools.

There are a few pieces involved in setting this up:

Importing the guest tools installer into our software repo.

Configuring this item as an install for our test machine or group of machines.

If we want to do this in a smart way, we can also have the management system install these tools only if the client is actually a VMware Fusion VM. By doing this, we don’t have to explicitly set tools to install on specific machines, and instead let Munki do this conditionally. Munki’s mechanism for this is called conditional items.

The version of Fusion I’m using at the time of writing this is 7.0.1.

Importing the guest tools

VMware Fusion keeps its various guest tools in /Applications/VMware Fusion.app/Contents/Library/isoimages. OS X tools, of course, live in the darwin.iso disk image. If this disk image doesn’t exist, you may be on a system that has never created an OS X guest and whose VMware Fusion distribution didn’t come with tools included. VMware Fusion will download the guest tools as needed if this is the case, so create a dummy OS X VM just to get Fusion to download the tools.

We can open this up and see it’s got a plain installer:

This isn’t actually an installer, it’s an application disguised as an installer. It seems to be a shim app that simply loads an embedded installer package using the standard Installer app. I’m not really sure why this exists – but it does. Luckily, if we dig into this bundle (right-click and “Show package contents”), we’ll find the actual installer at Contents/Resources/VMware Tools.pkg. This is what we want to tell Munki to import. We can use munkiimport to import this disk image interactively.

(An easter egg: If you have a VMware Fusion 6.0.x app bundle laying around, take a closer look at the folders at the top level of the mounted darwin.iso image. It contains a small set of additional bootloader files and drivers, some of which are used for legacy DVD-based installs.)

When Munki installs from items on a disk image, it can look for these items at arbitrary paths, meaning we can import the vendor iso file directly. This saves us the small step of needing to first mount the disk image and locate the real package, and we can more easily automate this process. Being able to use the vendor installer directly for almost all software is one of Munki’s many strong points as far as cutting down on tedious repo management, especially for cases where an application may have multiple installer pkgs within a single DMG (Autodesk Maya, for example).

Here’s the output of munkiimport (note the use of the -p option to specify the path to our installer, which makepkginfo will use to set the package_path key in the pkginfo plist file.

In this case, I’ve accepted the most of the defaults, given it the item name of VMwareFusion and put it in a repo subfolder of support/VMware.

Recent versions of this tools installer seem to use a sane package identifier and version number, so I haven’t needed to make additional changes to the Munki pkginfo. If you do wish to add an additional installs array to your pkginfo, one place where you will find components that are likely to be unique for each tools version are the kernel extension bundles in /Library/Application Support/VMware Tools/, for example the vmhgfs.kext bundle. These tend to also have the same CFBundleShortVersionString as the tools installer packages.

Note that the package version of 9.8.4 isn’t very meaningful to us, so you may wish to change the pkginfo’s version key to 7.0.1 just so that you know which tools package is for which version of Fusion. They do seem to at least increment in a logical fashion with new Fusion releases. All that matters here is that you are consistent, because when Munki attempts to install these tools it will pick the highest version it finds in the first catalog in which it is found.

Making it available to clients

The model of how Munki decides what software to offer to clients is simple. A client looks for a specific “manifest” file on the server, matching a ClientIdentifier that’s been configured on the client or, in its absence, several fallback values: the client’s FQDN, its “short” hostname, the Mac’s serial number, or a default manifest called site_default.

These manifests are plists containing an array of catalogs that will be searched for the client, and typically one or more arrays of containing installer items to be installed, updated (only if already installed), made available through an “Optional Installs” self-service interface, or removed.

While we could explicitly set this VMwareTools item to be installed on clients we know to be running in VMs, we can make this smarter and only process this install item if Munki determines that this client is in fact a VM. This way, we can define this in a manifest that may be shared by any number of VMs and physical machines. Munki can include manifests in other manifests using the included_manifests array key.

My testing clients (including my main workstation) all include a manifest called “utils”, which contains a list of software that’s useful for me to always have available on testing machines. This includes debugging utilities, command-line tools, and Mac admin tools that I find useful to always have at hand for testing. Since I’m including this manifest for all test machines anyway, I’d like to just add the VMware Tools to this same manifest, and have Munki figure out whether it’s needed. To do this, we’ll look at “admin-provided conditions” in Munki.

Writing an admin-provided condition

Conditions are Munki’s term for attributes of the client system that can be derived automatically every time Munki runs, and which it can use to conditionally determine whether certain items are installed. The equivalent in Casper is the Smart Attribute, or for Puppet, “facts” derived by the Facter tool. This is a common pattern among client/server management systems.

Munki expresses the conditions using Apple’s NSPredicate syntax, which allows us to define an expression using these conditions and which evaluates to either true or false. If true, whatever installs or removals are defined for that condition will apply to this client. Conditions can also be nested.

Munki includes some built-in conditions for attributes like the client’s OS X version, whether it’s a desktop or laptop, and more. There’s one called machine_model, which reports the model identifier (“iMac15,1″, etc.). Since VMware Fusion VMs use model identifiers like "VMware7,1", we could potentially use a condition that looks like: machine_model BEGINSWITH "VMware". For me this was not sufficient, because for certain VMs I make use of VMware’s ability to “spoof” different model identifiers and test some conditions in a way that better simulate running on physical hardware. Since there’s nothing else built-in to Munki I could use for this, I went the route of writing my own condition.

Munki supports these additional conditions in that it will run any executable files located in clients’ /usr/local/munki/conditions directories. These executables are expected to populate values in a ConditionalItems.plist file in the Managed Installs directory that Munki uses for its data. These are frequently simple scripts that run some system command and extract the data from the output of the command. The data that were interesting in deriving here is simply whether this client is a virtual or physical machine.

I’ve written a basic condition script that provides a value for a condition called virtual, and it’s posted here on GitHub.

Copy this script to a VMware Fusion guest already configured with Munki and able to get updates from a manifest. Make sure the script is placed in /usr/local/munki/conditions, is executable, and not world-writable.

Now we can define a new block in our manifest for this client, using the conditional_items array. Here’s an example of a complete manifest including one conditional item:

And with this, we’ve seen how we can import VMware guest tools into Munki, and take advantage of Munki’s admin-provided conditions to dynamically install them on clients that can make use of them.

]]>http://macops.ca/keeping-your-os-x-vm-guest-tools-current-with-munki/feed/0More about suppressing diagnostics submissions popups in OS X Yosemitehttp://macops.ca/diagnostics-prompt-yosemite
http://macops.ca/diagnostics-prompt-yosemite#commentsTue, 18 Nov 2014 19:35:17 +0000http://macops.ca/?p=801With OS X Yosemite, Apple added an additional phase to the Setup Assistant: the offer to submit diagnostics info to Apple and third-party developers, which is displayed either as part of a initial setup or upon first login (similar to the iCloud prompt).

Those who administer OS X clients typically look to disable such prompts on managed machines, either to avoid annoying users in shared workstation environments or because the organization may not (or may) wish to provide diagnostics information to Apple and third-party developers.

Both Rich Trouton and myself have documented what seemed to be an additional preference key that could be configured in the com.apple.SetupAssistant domain: LastSeenBuddyBuildVersion. However, with the release of OS X 10.10.1 on November 17, some admins reported seeing this dialog pop up again, and then that it might be possible to suppress by updating this new key with the updated build number of OS X 10.10.1, 14B25.

Furthermore, whether it would show up seemed it may depend on whether the user is an admin or not. If the user was not an admin, the setup assistant window would still show but would simply show the “Setting Up Your Mac..” animation that plays at the end of the setup assistant process.

Back when Yosemite was available only as developer previews, Rich had already documented on the Apple dev forums a process that seemed to disable this diagnostics prompt. This involves writing additional keys to a file at /Library/Application Support/CrashReporter/DiagnosticMessagesHistory.plist. In my testing, unchecking both checkboxes (Apple and app developers) for diagnostic submissions results in at least the following keys getting set in this plist:

I looked again at whether this was still something that comes into play given this most recent 10.10.1 update. Digging through the binary at /System/Library/CoreServices/SubmitDiagInfo seems to suggest it is, with logging messages like: Diagnostic message history store was not writeable. Will not submit diagnostic messsages, admin user was unable to write into diagnostic message history, and methods that determine whether the authenticated user is an admin user. This all confirms that the service managing the diagnostic messages expects that admin users can write directly to this file (and indeed, systems I’ve seen all set this file to have read/write access for the admin group).

I’ve since performed tests deploying an new, unbooted 10.10.1 image that contains no LastSeenBuddyBuildVersion key in com.apple.SetupAssistant, where in previous Yosemite testing I’d been setting this key via a Configuration Profile.

So as far as I can tell, it may be enough to suppress this diagnostics prompt using only a DiagnosticMessagesHistory.plist file placed at /Library/Application Support/CrashReporter/DiagnosticMessagesHistory.plist, containing the above four keys. I’ve tested deploying this file within an image (built with AutoDMG) using a standard installer package with no scripts.

One could also apply these plist keys to a booted system (using Munki, for example) using a script like the following. Note the lack of the "$3" variable, meaning this script would not apply to non-booted volumes if run within a postinstall script. This script actually leaves the defaults as suggested by Apple, so tweak as desired – the objective here is to set them to something so that this phase of the Setup Assistant does not show.

I consider this all still speculative. Rich Trouton has (also today) documented an alternate approach to suppressing this diagnostics dialog. My theory at this time of writing is that while perhaps updating the setting for LastSeenBuddyBuildVersion in the Setup Assistant prevents these additional screens from showing, it’s not what is actually determining the behavior of the diagnostics reporting mechanism.

]]>http://macops.ca/mactech-deployment-discussionbofqa-notes/feed/2How Do I Contribute? MacTech 2014 presentation linkshttp://macops.ca/mactech-2014
http://macops.ca/mactech-2014#commentsWed, 05 Nov 2014 22:20:44 +0000http://macops.ca/?p=775For everyone at the MacTech Conference in Los Angeles this year, here are links to various resources that are linked and referred to in my talk today on Git and source code collaboration.

]]>http://macops.ca/mactech-2014/feed/0AutoPkg: Crowd-sourcing Mac packaging and deploymenthttp://macops.ca/autopkg-msa2014
http://macops.ca/autopkg-msa2014#commentsWed, 17 Sep 2014 13:53:57 +0000http://macops.ca/?p=742Thanks to everyone who attended Greg’s and my talk today at MacSysAdmin 2014 in Göteborg! Again, I give my sincere thanks to Tycho and all those who organize the fantastic MacSysAdmin conference every year. I’m honoured to be among such great company, speakers and attendees.

]]>http://macops.ca/autopkg-msa2014/feed/0A tour of Charles, your HTTP(S) Swiss Army knifehttp://macops.ca/charles
http://macops.ca/charles#commentsTue, 22 Apr 2014 13:29:33 +0000http://macops.ca/?p=682There are times when it’s helpful to be able to know exactly what HTTP traffic is being sent or received on your Macs. Perhaps you’re auditing a 3rd-party application to see what connections it makes to outside servers, or maybe you’re interacting with – or writing – a REST API. Perhaps you just want to see every transaction between you and Apple’s servers when you use the Mac App Store to download apps, or use Internet Recovery.

Anyone doing systems administration long enough will have eventually used the packet capture library in some form, usually in the form of tcpdump and/or the Wireshark application, a powerful set of tools for analyzing all types of network traffic. This is very useful if you’re writing a NetBoot server replacement and need to inspect at the packet level, but if we’re only interested in HTTP(S) traffic, there are better, more specialized tools available. In this post I’ll introduce Charles, a web proxy and GUI tool for inspecting and diagnosing HTTP traffic. Since I’m ofteninterested in knowing how software performs update checks, I’ll use this as an example.

For a simple example for getting familiar with Charles, let’s say we want to know how an application checks and notifies the user about new updates. This would be useful to know if we’d like to write an AutoPkg recipe that’s able to automatically download the latest version of an application. We’ll look at the Adium instant messaging client. Note that these URLs will eventually be out of date and may not work to try on your own, but hopefully the output here will illustrate our work well enough.

Like many other applications on the Mac, Adium uses the Sparkle framework to handle the application’s built-in updater mechanism, which retrieves update information from an RSS feed URL. AutoPkg, conveniently enough, has a processor that’s able to take a Sparkle RSS URL as input to its downloader processor, so we don’t need to write our own parsing mechanisms. This RSS URL is usually given in the application bundle’s Info.plist file in the SUFeedURL key. In this case, it’s http://www.adium.im/sparkle/update.php. So, what do we need Charles for? Can’t we just feed this URL to AutoPkg? Let’s test it with cURL:

What’s going on? Version 1.1.4 is old. The most recent version of Adium at this time of writing is 1.5.9, and if we’d go and actually check for updates in Adium, it would report that it’s at the latest version. We need to see exactly what’s going on, and this is where we can use Charles.

Charles is a paid software, but it also functions as a time-limited trial, which is more than enough to get our feet wet. The first time you launch Charles, you’ll get a couple prompts. One will be an an admin user prompt so that Charles can register the rights to manage your system’s network proxy settings. This allows Charles to automatically configure the local system’s HTTP and HTTPS proxy to point to itself (by default using port 8888). This feature is extremely convenient. Launch Charles and immediately see traffic flowing through it. Close Charles, and the system is automatically reconfigured as it was before.

Another dialog you’ll see on first launch is the offer to help configure Firefox’s proxy settings, which are independent of the system’s. Some other applications will use their own proxy settings rather than those managed in OS X’s Network settings – command-line tools like cURL require setting the http_proxy and https_proxy environment variables, for example.

With no other configuration, you should start seeing some traffic showing up in Charles on the left-hand sidebar, as Charles by default begins recording HTTP traffic in a new session automatically when it is launched. The traffic you see comes from whatever applications may be running on your computer, website sessions, etc. Now we can open up Adium, choose “Check for Updates” in the Adium menu, and look at what Charles has saved for us.

Adium checking for updates. Note the HTTPS redirect, but no recorded data in the HTTPS URL.

Sure enough, there’s the request to http://www.adium.im, and we can open the disclosure triangles that open with each path component in the URL. We can see the final part of the path, and also click on it to copy the full URL: http://www.adium.im/sparkle/update.php?generation=2&amp;type=release. This is what the client actually requested. Try checking the “Update to beta versions when available” and “Include anonymous system profile” preferences in Adium and then see how the request’s query string changes. Explore the different tabs on the right hand view, particularly the different views for the request and response; here we can see each query string broken down. Ever wonder what those “include system profile” settings in applications are actually sending? This is how you can find out!

In fact, we can even edit the request details right here and re-execute them, which can be especially useful if you’re experimenting with a REST API:

Editing request data right within the recording.

During this time, if your window is getting cluttered with other requests, you can clear them from the recording with the Trash can icon, selectively delete them, or open the Proxy Menu -> Recording Settings and add something like *.adium.im into the Include Locations, which will filter out all other traffic. Don’t forget to clear it later if you do.

Going back to our initial results, the response was actually a 301, a permanent redirect to an https:// URL. Charles doesn’t proxy SSL by default, so if you look at the https://www.adium.im:443 URL that appears immediately after this request, in the Overview tab, you will see “SSL Proxying disabled in Proxy Settings.” You can still look at the response, but it’s just the encrypted HTTPS traffic, not much use to us here. We’ll come back to configuring SSL in Charles, but let’s quickly use cURL to check the redirected response. You won’t see this traffic in Charles unless you set http_proxy and https_proxy environment variables to localhost:8888 or 127.0.0.1:8888, but we don’t need to bother with that here.

The curl command, again following redirects and quoting the URL to prevent the shell from interpreting characters like ?:

This looks better. We can see that there are both new and older versions, targeted towards clients running older versions of OS X.

We can stop here, because we now know exactly what Adium sends to its update server in order to check for new versions, and we can write an AutoPkg recipe to do the same thing. Of course, one already exists, and you can see how the recipe integrates these additional query strings in the region highlighted in that link.

But we’ve gotten this far; how would we have inspected the encrypted data in Charles if we’d needed to? First we must configure a root certificate authority to be trusted by our system, so that the certificate used by Charles’s SSL proxy can be verified in the handshake process. Charles also makes this easy: From the Help menu, choose “Install Charles CA SSL Certificate…”. This will open the Keychain Access application and prompt you to trust this in the system.

Charles’s SSL Proxying cert after it’s been added to Keychain.

You can also obtain this same certificate from the Charles web site at http://www.charlesproxy.com/ssl.zip, if you are ever setting up another client to proxy to Charles, and install it with the security command, which for this will require root privileges:

In addition to this, we need to tell Charles to actually proxy SSL requests, which it does not do by default. We can set this in the Proxy Settings menu. This setting also requires us to specify a host filter, which can be a wildcard *, but could also be limited to the domain we’re interested in. You may find that with it enabled and set to all URL domains, that some services cease functioning due to the use of an alternate SSL CA.

We conclude this tour of Charles – if you’ve read this far, it should be obvious that this tool is well worth its $50 price tag and that it’s is useful for both simple uses (as in this example) and the many advanced features that we haven’t explored in this post.

]]>http://macops.ca/charles/feed/0Building native extensions since LLVM 5.1http://macops.ca/building-native-extensions-since-llvm-5-1/
http://macops.ca/building-native-extensions-since-llvm-5-1/#commentsTue, 15 Apr 2014 16:22:36 +0000http://macops.ca/?p=667With LLVM / clang 5.1, Apple introduced a change where any unrecognized command option causes a hard failure. Unfortunately, there are many packages in the Python package index that have not yet adapted to this change when building on OS X and include unsupported flags (in my experience it’s usually been -mno-fused-madd). I first started running into this frequently when installing some Python packages using pip. This can also be an issue for other package managers like RubyGems.

clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
error: command 'cc' failed with exit status 1

Sometimes packages will still install and fall back to slower, non-native packages. Some packages, like lxml, simply won’t install.

Luckily there is a straightforward workaround: define the CFLAGS environment variable and pass it the compatibility option -Wunused-command-line-argument-hard-error-in-future when we execute the command. The C compilers look for this and include these additional arguments. Like this:

]]>http://macops.ca/building-native-extensions-since-llvm-5-1/feed/0How to Package Profileshttp://macops.ca/how-to-package-profiles/
http://macops.ca/how-to-package-profiles/#commentsThu, 10 Apr 2014 20:51:38 +0000http://macops.ca/?p=653Part of a managed Mac’s configuration is often one or more Profiles, either Configuration Profiles, or an Enrollment Profile for an MDM server like Apple’s Profile Manager or Cisco Meraki Systems Manager.

There are multiple ways to install these. You can have users double-click and install these .mobileconfig files themselves via a website or e-mail if they have administrative rights on their machines. You can have DeployStudio install them as part of a workflow and not care how it’s done, or have a management service like the Casper Suite configure and manage them for clients (and again, not need to care how it’s done).

But if we want the most portable way possible to install a profile on a Mac, it might be the simplest to fall back to the Mac’s lingua franca of software configuration (for better or for worse): the Installer Package. A profile that can be installed via a package install can be installed with any management software (even Apple Remote Desktop, version 3 turning 8 years old tomorrow). And like a profile, it can simply be opened and installed like any other piece of software by a technician or user.

This question comes up often enough for people using Munki, or other management systems that don’t have some kind of purpose-built mechanism for dealing with profiles. Generally these are the steps:

Have the package install the profile somewhere like any other file.

Run the command /usr/bin/profiles (as root) in a postinstall script to install this profile.

There’s no step three.

Actually, there is if you’d like to also include a mechanism to remove the profile: You’d want to write some short script that would remove the profile, as well as configure the system so that it can know that this profile is no longer installed. Since we’re using an installer package, we have the benefit of being able to check for a receipt of the package.

You also might like to install this profile on an non-booted volume (either a clean image built with something like AutoDMG, or a Mac system connected via Target Disk Mode). But since we don’t have profiles available, we actually want to install it to a special place that OS X looks for at boot time for any profiles to install: /private/var/db/ConfigurationProfiles/Setup, as well as clear a special .profileSetupDone file that may exist if this volume has been already booted. This has been documented already in a fewplaces. If you build images for deployment, you may have scenarios in which it’s important for the profile to be installed on the system’s first boot rather than later in its software management cycle.1

I got tired of fiddling with these details and making errors copying and pasting profile packaging/uninstall scripts from one profile to another, so I wrote a short Python utility to make this easier to automate. It’s called “make-profile-pkg”, and it lives here on GitHub. There are more details on the how and why there on the GitHub page.

Graham Gilbert contributed a couple great things on this: He helped make it a generic pkg-building tool rather than a Munki-specific tool, and also added the logic in the postinstall script for it to do the right thing about the install location depending on whether the package is used on a booted or non-booted volume. This allows a single package built with this utility to be installable on both booted and non-booted volumes. In addition to building the package with a few configurable options, it will also generate an uninstall script that can be used in conjunction with your software management platform of choice.

As of December 12, 2014, the tool also generates an installcheck_script that will be used by Munki to check whether the profile is actually installed, since in some cases it may be possible for a user to remove the profile after the package has installed it. The script used here is from Graham’s MacTech 2014 session on using Munki for client configuration.

An important distinction here is to set the package to use the full path starting with /private/var rather than simply /var, which is actually a symlink. It’s been reported to have caused issues in some cases before, and it’s simply not correct: install files to real paths rather than symlinks whenever possible. ↩

]]>http://macops.ca/how-to-package-profiles/feed/1Installing Command Line Tools automatically on Maverickshttp://macops.ca/installing-command-line-tools-automatically-on-mavericks/
http://macops.ca/installing-command-line-tools-automatically-on-mavericks/#commentsWed, 23 Oct 2013 13:01:01 +0000http://macops.ca/?p=486In Mavericks, the Xcode Command Line Tools can be downloaded from the ADC downloads page like with previous versions. Now, though, they can be also be installed on-demand in a similar fashion to how Java has been installed since Lion, by simply invoking a command installed by them such as otool, or a new option in the xcode-select utility: --install.

In this post we’ll look at how you can trigger and run this installation in an automated way, eliminating the need for any user interaction.

We can guess from the dialog window that this mechanism might use Apple’s Software Update servers as the source of the installer. You’d also already know this if you run your own Software Update service or Reposado (which is also included in JAMF Software’s NetSUS appliance), and could have been syncing the Mavericks SUS catalog since June.

This task is a great opportunity to get familiar with how you can use an update’s .dist file to identify what criteria makes an update “available” when the Software Update framework is invoked on a client.

Reposado makes this easy: get the list of updates with repoutil --updates, and note the item for the CLI tools:

Now print out an update’s dist by passing its ID to the --dist option:

./repoutil --dist 031-1006

We get back a bunch of XML, much of which is JavaScript code that the client will run in order to evaluate whether this update applies to it. Here’s a section that looks interesting (and is even commented more than usual):

function isVisible(){// Must have a prior version of CLTools_Executables installed, or have the file marker that indicates and install-on-demand is in progress.var receipt = my.target.receiptForIdentifier('com.apple.pkg.CLTools_Executables');if(null== receipt){// No receipt found for CLTools_Executables, check if the IOD application// is running. We do this by expecting the IOD application to create the// temporary file we check below for existence.if(system.files.fileExistsAtPath("/tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress")){returntrue;}else{returnfalse;}}

The trigger mechanism is spelled out here. There are other checks within in this .dist’s <script> tags that must be satisfied, for example the OS must be 10.9, and the user must not have already the receipt for this package present on the system. But here we can see the extra bit that the GUI helper application does for us: it touches a temporary file at /tmp/.com.apple.dt.CommandLineTools.installondemand.in-progress so that when functions in the Software Update framework are called, the helper application can find and install the appropriate update.

Similarly, Rich Trouton has also (of course) previously documented the trigger necessary to do an automated install of the latest Java 6 update available in Software Update, which uses an environment variable instead of a trigger file.

The tags are something that the Software Update framework or helper tools can parse to filter out candidate update packages. These are usually used by system tools and preferences other than softwareupdate itself, for example for finding Boot Camp ESD packages, printer drivers and speech assets.

Now that we’ve taken a side-tour of Software Update .dist files and how you can make use of them to identify how OS X software updates are parsed and selected, let’s get back to why we’re here: to install the CLI tools on Mavericks (and future OS X versions that will hopefully continue to use this mechanism). Now that we know the trigger file needed, the rest is straightforward.

A simple, though not exactly clean, way to do this is to scrape the output of the softwareupdate utility to extract the label for the update we want. I have a general-purpose script for installing Xcode CLI tools on 10.7 through 10.9 here.

It looks for the section of text we’re interested in like this:

Software Update found the following new or updated software:
* 031-1006-5.0.1.0
Command Line Developer Tools for OS X Mavericks (5.0.1.0), 99103K [recommended]

By looking for the word “Developer” and then passing softwareupdate the label next to the asterisk in the line above.

This is less than ideal, because if either the wording of the update, or the command output formatting of softwareupdate were to change, this tool would break and we would need to fix it.

An better alternative might be to look at the contents of /Library/Updates/ProductMetadata.plist after running softwareupdate -l, and search the tags that were already extracted for us in a nice structured format. We find an item like this:

Unfortunately, the “label” that softwareupdate can take to install a specific update still needs the longer version (“031-1006-5.0.1.0″), so we’d need to do even more work to do this using only structured data.

One way would be to look at the com.apple.SoftwareUpdate any-user defaults domain and the RecommendedUpdates key:

We could find our ID in the Identifier sub-key, then take the value o that item’s Display Version key, then feed both of these to softwareupdate joined by a hyphen. This would eliminate the need to do any kind of parsing of the softwareupdate command output.

Because all this work would require sifting through plist contents, it’s probably no surprise that I’d recommend Python to do this; besides having easy support for working with plists mapped to native data structures, on OS X it can also natively access the com.apple.SoftwareUpdate preferences domain using Objective-C (via the PyObjC framework). So, this is left up for either a rainy day and/or an exercise for the reader. In the meantime, the less-than-ideal shell script version works.

]]>http://macops.ca/installing-command-line-tools-automatically-on-mavericks/feed/3Deploying Xcode and CLI tools: what’s new in Xcode 5http://macops.ca/deploying-xcode-and-cli-tools-whats-new-in-xcode-5/
http://macops.ca/deploying-xcode-and-cli-tools-whats-new-in-xcode-5/#commentsThu, 19 Sep 2013 19:17:22 +0000http://macops.ca/?p=586Xcode 5 was released to the public on September 18 along with iOS 7. If you deploy Xcode and the command-line tools, a few things have changed since 4.x. There’ve been a couple other posts on this blog in the past about the steps required to successfully deploy Xcode and/or CLI tools.

In this post we’ll look at what’s new with Xcode 5.

Review

The dvtdownloadable index (Apple’s metadata plist-based feed for Xcode additional downloads, used by the “Downloads” area of Xcode’s Preferences) remains at its same location:

You can monitor this URL for changes to find out when new components are available, as well as determine under what conditions they’ll appear, how they are considered installed, etc.

The app still includes two device development-related packages that absolutely must be installed in order for Xcode to function. They still have the same meaningless package version numbers, so there’s no easy way to know which version is present on a system besides manual installation of the packages included with a specific Xcode version. Automating the installation of these packages after copying Xcode.app to /Applications still seems the most sensible option here.

Accepting the license

I don’t recall ever needing to do this before, but this needs to be done and with root privileges:

If this isn’t done before first launching Xcode, it will prompt for admin rights to persist this change on the system (after displaying the EULA).

CLI tools

It’s a little-known fact that for the past couple of years, Xcode’s CLI downloads don’t actually require ADC authentication and are accessible via public URLs. Update: This seems to be still the case. Michael Kuron replied to this post below, with the info that as of a few hours after this was originally posted, that the CLI tools still don’t require ADC access, which is good news.

Related, Xcode 5 contains a new Accounts management system that enables certain tasks that would otherwise be done via the ADC portal to be authorized and done within Xcode itself.

We can likely expect the state of getting CLI tools to change as well with Mavericks. See the “Developer Tools Install-On-Demand” feature listed at bottom of Apple’s Mavericks preview page.

Some docsets also require ADC access

I’m pretty sure no docsets for public docs ever required ADC authentication, but now the 10.8 docsets do. It is still possible to install these without admin rights because there is a special authorization right that exists specifically for this: com.apple.docset.install.

To get a list of docsets and their actual download URLs in the same format at the other DVT downloads, check out the feed at this URL:

These docsets ship as standard packages, however they are similar to those for iOS simulators in that they do not include a target location in the pkg metadata. If you’d want to deploy these packages to their correct location as would be when handled by Xcode, you’d need to repackage them as I’ve outlined in this post.

The Oracle Java 7 JRE (a web plugin) began shipping last year, and has grown a small maze of clever mechanisms to maintain a schedule of checking for updates. It’s a sad tale of the misuse and abuse of launchd schedule re-writes and re-loads, the Sparkle Framework, storing Java properties-like prefs in OS X defaults, and having two different systems that actually check for updates implemented in two different languages and runtimes.

The takeaway from those previous two posts is that the plugin has a mechanism triggered by the applet to check for updates, but because this only runs once the plugin is loaded via a browser, there is also a background-check LaunchAgent that prompts the user to install the latest version via a Sparkle dialog (a process which later goes and re-loads LaunchAgents as root instead of you, but read the earlier blog posts if you care.)

Now that Update 40 has been out for over a week, I’ve taken some time to look at the changes to the installation that should be of interest to anyone deploying it en masse.

-bgcheck

The background-check LaunchAgent at /Library/LaunchAgents/com.oracle.java.Java-Updater.plist (actually symlinked to a location inside the plugin’s bundle) now runs the Java Updater binary with a new flag: -bgupdate, and this respects a preferences key JavaAutoUpdateEnabled. Full credit for picking this up goes to kbotnen in the ##osx-server IRC channel.

We can make use of the JPI_PLUGIN2_DEBUG debug environment variable to get some console output to test this assertion:

com.oracle.*

So this takes care of the background updater. From now on, it should no longer be necessary to do silly tricks like unloading and removing symlinks after every installation just to disable it. But for the Java applet itself performing update checks on loading, we’d like to be able to disable these too.

I’m no Java developer, but as far as I know it typically uses XML or ‘properties’ files to store data (the latter especially for configurations or ini-like options). If I look at filesystem changes when unchecking the “Check for Updates Automatically”, I eventually see a change show up in my user’s folder at ~/Library/Preferences/com.oracle.javadeployment.plist. This is essentially a plist representation of a the properties file, and the key of interest is deployment.macosx.check.update located within the /com/java/deployment/ dict key.

Going back a step, this key also gets set automatically when the Java Control Panel loads and there are 1) no prefs yet in the com.oracle.javadeployment domain but 2) the JavaAutoUpdateEnabled key is set in com.oracle.java.Java-Updater. So it’s possible that it’s no longer needed to manage anything in com.oracle.javadeployment. However, it only gets populated if the Control Panel is launched, not on a regular loading of the web applet. It’s hard to tell whether this matters, because testing this on the latest release obviously won’t prompt about a new update being available.

If we did want to pre-set this properties-style pref anyway, it seems we can put this plist into the any-user domain at /Library/Preferences/com.oracle.javadeployment.plist instead of the user’s, and we can whittle it down to the essential key:

It’s possible that without adding some of the version-specific properties settings the applet may still warn about outdated versions, but I really prefer to not add version-specific tweaks with every update, even if those tweaks can be automated – they’re just one more thing to break and keep track of.

Surprise! The installer has gotten worse

The bad news is, the postinstall script has grown some new warts. Here’s the last section of the postinstall script that’s new:

# Launch verify Java URL, this script must be at the end of this fileif["$COMMAND_LINE_INSTALL" = ""]; thenDEFAULT_BROWSER=$(GetDefaultBrowser)`${OPEN}-gb${DEFAULT_BROWSER}'http://java.com/verify/?src=install'`fi

Note the install.jar file referenced up on line 6, which is executed by the newly-installed Java runtime. The java.settings.cfg file doesn’t get installed by the installer payload, but is assumed to exist when invoking this command. Maybe this is optional support for a deployment configuration file that can customize the installation. Either way, the script doesn’t care to check whether it exists, and so this command will spit out a Java traceback in your install.log and happily continue on. Lucky for us this, isn’t the last command executed in the script. Speaking of, what _is_ the last command in this script? It used to be launchctl commands that naively assumed they would always succeed in loading a LaunchDaemon. See the line at the end beginning with `${OPEN}.

We all love installers that call open after the installation run has finished, but they’ve at least done the right thing of checking for the existence of the COMMAND_LINE_INSTALL environment variable set by the installer framework when doing command-line installs.

But nested within that is the baffling GetDefaultBrowser function that seems to try to figure out your user’s default browser setting by combing through the LaunchServices plist and defaulting back to com.apple.safari. My immediate reaction that this function would exist for handling something like Google Chrome, which has no support for the Java plugin as it’s not yet 64-bit. But no, it just wants to know the bundle ID associated with the http URL scheme, so that it can pass it over to open -b, rather than trusting that open would do the right thing (use the default URL handler) with the http:// URL being given to it.

On my system, where I don’t install packages as my own user (because I’m not an admin), this failed when it managed to find a downloaded Chrome app bundle in my admin user’s Downloads folder and tried to open the URL with that. This could similarly fail if you happen to do one-off installs by a local admin user on a desktop support call, if your admin user happens to have some history on the computer. Because this command on line 33 is the last command run in the script, the postinstall command exist non-zero and the install fails with no evident reason why, until I check the install log, see the first traceback error and realize I can safely ignore it, and then find the real cause of the error.

Deploying Java via a management utility like ARD, Casper or Munki should luckily always bypass this function invocation in the script altogether via setting COMMAND_LINE_INSTALL, but this is simply extremely poor form. No one should be putting scripts like this in a plugin installer that’s widely used. Rich Trouton has helpfully documented where you can go to file a bug report to Oracle.

]]>http://macops.ca/java-7-web-plugin-deployment-redux/feed/1Configuring ColorSync display profiles using the command-linehttp://macops.ca/configuring-colorsync-display-profiles-using-the-command-line/
http://macops.ca/configuring-colorsync-display-profiles-using-the-command-line/#commentsFri, 16 Aug 2013 13:45:10 +0000http://macops.ca/?p=514Managing ColorSync ICC profiles for displays is something I do for certain workstations via MCX, and it’s always been a pain. Typically I would manually configure a profile for a display, then open up that user’s ByHost .GlobalPreferences preference stored on disk, extract the keys for the hardware-specific GUID that corresponds to that monitor (something like Device.mntr.00000610-0000-9C6B-0000-000004271AC0), and import them into MCX, ending up with a blob like this:

…which works, but I had to do a lot of manual work to get this configured, and I don’t want to repeat this for 1) every machine needing a managed profile, and 2) every time a Mac or display gets changed. It would be nice if we could just specify a profile on the command line for a given display, and make it so.

Sure enough, there is a supported ColorSync API that can handle this, and the PyObjC Python-Objective-C bridge is there to help us implement it with little code, and no Xcode project or compilation required. I wrote a simple command-line utility that I’ve put up on GitHub here.

It turns out that this preference can also be configured at the “any-user” level, so this tool supports that. There’s also a sample helper script in the GitHub repo that demonstrates how you could run this run this utility at login time for all users, such that these profiles can be managed easily by those calibrating the monitors.

]]>http://macops.ca/configuring-colorsync-display-profiles-using-the-command-line/feed/0Java 7: How not to use launchd for your apphttp://macops.ca/java-7-how-not-to-use-launchd-for-your-app/
http://macops.ca/java-7-how-not-to-use-launchd-for-your-app/#commentsFri, 15 Mar 2013 22:03:06 +0000http://macops.ca/?p=418The Oracle Java 7 package contains launchd items to support its Sparkle-based background update check app that I complained about previously. In this post we’ll go through its logic exhaustively and use it as an example of how to not deploy a LaunchAgent, and issues when trying clever things in LaunchDaemon scripts.

For some, there should be new information about how launchd works in general, as I think for many admins its behavior is somewhat opaque. Along the way I also learned some new launchctl command options.

Introducing ‘Helper-Tool’

First, let’s paste the entire Helper-Tool script, and go through it. This is called by the com.oracle.java.Helper-Tool LaunchDaemon, and is triggered whenever the com.oracle.java.Java-Plugin LaunchAgent plist (which is actually a symlink to a plist in the plugin’s Contents/Resources directory) is modified.

#!/bin/bash# This is a specialized randomizer function# that will randomize when AU will be triggered# for sceduled updates for a Mac
rand(){localmax_value="$1"n=$RANDOMvar=$[1 + $n%$max_value]retValue=$var}

Two problems. One, ‘/usr/libexec/PListbuddy’ is a typo. It just goes unchecked because probably 99.9% of OS X systems are on a case-insensitive HFS+ filesystem, but OS X fully supports installation onto case-sensitive filesystems.

Two, defining commands’ absolute paths using `which command` is useless. The which command works by searching the PATH environment variable for the executable. If you depend on being able to locate an executable by `which` in your script, you can skip pretending you’re using absolute paths, because you’ve already assumed they’ll be located in your PATH. You can get the default PATH used by launchd with the command: launchctl getenv PATH.

If any of the HT variables are undefined (-z tests for a zero-length string), then store a random value in retValue, and use the sed command to perform an inline replace it in the plist.

One might ask, if this system is designed specifically to “reset” the LaunchAgent schedule after an update, why not simply put this logic into a postinstall script instead, and set the schedule to something like once per day?

Using sed to modify a plist is just silly. Plists are structured data, and there are tools, like PlistBuddy used in the following three lines, that were made for exactly this. The 00 minute and 09 hour values correspond to the values that were already in the LaunchAgent plist delivered by the installer payload. This sed command is also already performed by the installer script.

Despite all of these silly workarounds to update a schedule plist, there are also weak assumptions in the if statement:

If any of the HT* variables stored in com.oracle.java.Helper-Tool by the installer don’t exist, then it must be able to find exactly these stock StartCalendarInterval times in the com.oracle.java.Java-Updater plist, and that these wouldn’t somehow conflict with numerical values elsewhere in the plist (we’ll see later that if one enables debugging for this LaunchAgent, it will).

Two, PlistBuddy’s ‘Set’ command requires the key to already exist – it does not add a missing key as with the ‘defaults’ command. The logic used by this if statement is vague, because it’s assuming that if any one key in one domain com.oracle.java.Helper-Tool is missing, then we must be able to find all three of some other key in a different plist com.oracle.java.Java-Updater.

What’s worse, is that by littering these various preference domains with various keys that all seem to be related to a schedule but only coupled by flawed scripts, admins that may poke at these values attempting to shortcircuit its update check behaviour may just further confuse the roundabout logic used in these scripts. Of course, vendors don’t design their packages with poking in mind, but given that this concerns behaviour that most system administrators will immediately want to disable, it doesn’t help to make it as difficult as possible to disable something that’s usually trivial with other software, something even Adobe can document and support.

Continuing on…

${CHMOD}644"${LAUNCHD_PLIST_SRC}"

Why would this be necessary? Permissions should be normally only handled by the installer payload, except in very particular circumstances that can usually be avoided. Maybe it’s here because the permissions are actually wrong (mode 664, when they should be mode 644) in the installer’s payloads for both launchd plists.

launchd and Session Types

This brings us to the topic of launchd and “Session Types”. LaunchDaemons and LaunchAgents can run in several different Session Types – for example, the ‘Aqua’ Session Type is run in the context of a user that is currently logged in at the GUI. You may have noticed before that when manually loading and unloading jobs, that you need to be root in order to manage jobs that are running at the system level, for example LaunchDaemons located in /Library/LaunchDaemons.

‘LoginWindow’ is another Session Type that can be specified if a job should be loaded only while the system is at the login window. The LimitLoadToSessionType key can be specified in a launchd plist to restrict in which Session Types it would normally be loaded, but using the ‘launchctl’ command permits the job to be loaded in other contexts. If LimitLoadToSessionType is omitted, then the default of Aqua is used. So in a normal scenario:

Machine boots up, and loads the loginwindow.

LaunchAgents that are able to run in the LoginWindow context are loaded.

User logs in.

Jobs running in the LoginWindow Session Type are unloaded, and the jobs available to run in the Aqua Session Type as the regular user are loaded.

In most if not all cases, if the machine was asleep when the StartCalendarInterval time arrived, the job will run immediately upon waking. Even if the LaunchAgent job was somehow being loaded at the loginwindow (remember, this is to launch the Java Updater app), it would simply die and complain that no connection to the window server was possible. Actually, for this package, it won’t, because the job’s StandardErrorPath is set to /dev/null. More on that later.

But since Helper-Tool is running as root, its invocation of launchctl will load the job as root, and now, guess what? We have two separate instances of the LaunchAgent running. What do you think happens when the StartCalendarInterval time arrives?

LSMultipleInstancesProhibited doesn’t prevent it the app from running as multiple users!

So now Java Updater is being run twice, once as you and once as root. The LSMultipleInstancesProhibited would prevent it from launching twice as a user (perhaps to prevent runs over weeks on an idle system from spawning the alert multiple times?), but it won’t help here, when the alert is being launched as different users. Moreover, depending on how long the machine has been running without a reboot, there may be some time during which the job is running with two different times set in StartCalendarInterval.

Because Java Updater uses Sparkle, selecting “Skip this version” will set the SUSkippedVersion key in the app domain being used, which in this case is com.oracle.java.JavaAppletPlugin. The version is as it is defined in the Sparkle XML feed (which may or may not be equal to a bundle version key). Because it’s running as two different users, these Sparkle-related preference keys are now defined in two different user homes. In other words, skipping a version as a normal user means that it will still run again as root, until it’s skipped when it launches as root. (For what it’s worth, these Sparkle preferences can be defined at the system level, but managing SUSkippedVersion keys for this application gets to be a very tedious game of catch-up, and is not at all how the key is intended to be used).

Just to be sure that this is really happening, here’s the output of execsnoop, a DTrace utility that logs new processes as they occur. Notice the first execution is UID 0 (root) with a PPID (parent process ID) of 1, and the second is 501 (me) and a PPID of 329. The PPIDs correspond to the launchd manager process for the System and my user’s bootstrap namespace. (You can check these yourself with the launchctl managerpid command.)

This relaunching-as-root issue is moot once the Mac reboots, of course, because then the LaunchAgent will load only in the user’s Aqua context as usual. But with laptops, it’s not uncommon to go for weeks without a reboot, which is about how frequently there have been recent security updates are being released.

No logging

I mentioned earlier that the LaunchAgent’s StandardOutPath and StandardErrorPath are both set to /dev/null. You’re free to run the Java Updater binary yourself to mimic what would happen at the time the LaunchAgent job would run, but there’s not much useful output. There’s also a debug flag you can set in your shell environment if you’d like to see a bit more: set the JPI_PLUGIN2_DEBUG flag to something (it can be anything, it just must be set): export JPI_PLUGIN2_DEBUG=1. You’ll then see some output like this if you run it manually:

So if we’d like to actually debug and log the behavior of the LaunchAgent itself, we can remove the StandardOutPath and StandardErrorPath keys (they default to the system log) and define our own environment variables in the job by setting the EnvironmentVariables key like so:

Of course, as soon as you modify this LaunchAgent to help you debug this, the Helper-Tool job helpfully runs and resets your modified StartCalendarInterval values and mangles your debug flag, because it just so happens to be looking for the string “2” anywhere in the plist and sets it to a random day-of-week integer. When I was originally debugging this, I commented out enough of the Helper-Tool script to prevent it from resetting my changes to the plist. I’d then unload and load the LaunchDaemon.

Diagnostic self-obfuscation

While Oracle’s JRE package was clearly not meant to be consumed and scrutized in this manner by any user (or sane person), one has to seriously wonder why someone thought it helpful to go to such lengths to obfuscate the system’s own mechanisms, hiding all traces of useful logging and status info; compare to the verbose output of Google’s Keystone daemon during a background Chrome update. Setting aside the bizarre self-healing schedule – for something that probably should be nagging the user once a day to update, since Apple will block it the next anyway – it’s amazing how difficult the package even makes it to test and debug its behavior. It seems that the release engineer on this project was not interested in being able to debug and test this easily himself.

]]>http://macops.ca/java-7-how-not-to-use-launchd-for-your-app/feed/3Managing Xcode CLI toolshttp://macops.ca/managing-xcode-cli-tools/
http://macops.ca/managing-xcode-cli-tools/#commentsFri, 15 Mar 2013 14:54:43 +0000http://macops.ca/?p=453In a previous post on deploying Xcode components, I showed how the iOS Simulators are defined in a metadata file used by Apple, called dvtdownloadableindex, which is a binary plist containing information about all the “Components” available in the Downloads preference area.

What’s useful about this file is that it describes in a human-readable way what Xcode uses to determine what component updates are available and what’s already installed. Up until yesterday, the CLI tools used only SHA-1 sums on specific binaries and libraries to determine whether the package was installed, which was somewhat frustrating to those of us deploying it, because it meant the actual package receipt version numbers were next to useless. Munki, for example, couldn’t use these to determine installed status, but one could at least use these to know what files to use to track the installation. Munki can use MD5 checksums to specify a file’s contents.

And here’s the full entry for the latest version (4.5.9) of the CLI tools for Mountain Lion, released yesterday on March 14, 2013:

<dict><key>dependencies</key><array/><key>fileSize</key><integer>118401880</integer><key>identifier</key><string>Xcode.CLTools.10.8</string><key>name</key><string>Command Line Tools</string><key>source</key><string>http://devimages.apple.com/downloads/xcode/command_line_tools_for_xcode_os_x_mountain_lion_march_2013.dmg</string><key>userInfo</key><dict><key>ActivationPredicate</key><string>$MAC_OS_X_VERSION >= '10.8.0' && $MAC_OS_X_VERSION < '10.9.0'</string> <key>InstallPrefix</key> <string>/</string> <key>InstalledIfAllReceiptsArePresentOrNewer</key> <dict> <key>com.apple.pkg.DevSDK</key> <string>10.8.0.0.1.1306847324</string> <key>com.apple.pkg.DeveloperToolsCLI</key> <string>4.6.0.0.1.1362189000</string> </dict> <key>RequiresADCAuthentication</key> <false/> <key>Summary</key> <string>Before installing, note that from within Terminal you can use the XCRUN tool to launch compilers and other tools embedded within the Xcode application. Use the XCODE-SELECT tool to define which version of Xcode is active. Type "man xcrun" from within Terminal to find out more.

Downloading this package will install copies of the core command line tools and system headers into system folders, including the LLVM compiler, linker, and build tools.</string> <key>Xcode.SDKs</key> <array/> </dict> <key>version</key> <string>4.5.9</string></dict>

Notice we have a download path, it doesn’t require authentication to the Apple Developer Center (not long ago they did), and note in particular the InstalledIfAllReceiptsArePresentOrNewer key, with a dictionary of package receipts and versions. If I generate a new Munki pkginfo for this installer, my receipts array looks like this:

…which is much nicer than managing a bunch of file checksums. Hopefully they start using only receipts from now on.

]]>http://macops.ca/managing-xcode-cli-tools/feed/0Everything you’ll wish you didn’t know about disabling Java 7 updateshttp://macops.ca/everything-youll-wish-you-didnt-know-about-disabling-java-7-updates/
http://macops.ca/everything-youll-wish-you-didnt-know-about-disabling-java-7-updates/#commentsMon, 25 Feb 2013 01:11:38 +0000http://macops.ca/?p=406Oracle’s Java 7 JRE for OS X was first officially released in October 2012. As expected, there have been issues deploying and testing it, amidst confusion about Apple’s Java 6 updates and it disabling symlinks to the web plugin, the pre-emptive disabling of Java with XProtect, and more.

And of course, the first thing administrators need to verify is that deployed software won’t periodically nag the user to install an update that they don’t have sufficient rights to install, or that they shouldn’t install for other reasons. I’ll cover a few ideas in this post specifically about the updater mechanisms and approaches to disabling it, and focus on other specific issues with this package in future posts.

Java uses what’s known as a “Java properties” format to store preferences, that looks similar to a .ini file. Many user preferences seem to touch a file at /Library/Application Support/Oracle/Java/Deployment/deployment.properties. This file’s options are somewhat documented.

In more recent versions, some settings (perhaps those known to have OS X-specific implementations, like the boolean deployment.macosx.check.update) seem to be stored instead in the defaults domain com.oracle.javadeployment in a Java-style namespace:

It doesn’t actually matter for now where the deployment.macosx.check.update setting is actually stored, because it has no effect on whether or not Java 7 will check for updates.

Update 13/03/15: It turns out that this does have an effect, it controls the plugin’s own built-in update checker, that I managed to never see while scrutinizing the Sparkle-based updater. So while it suppresses the nag when the plugin is actually invoked, it doesn’t control the background update check mechanism that’s detailed below and in a related post.

What? Oh.

There’ve been a fewdiscussionthreads about configuring the update setting, but all these attempts seem to do is tell the Control Panel about the state of the checkbox, and not suppress the active checking (and prompting) for updates.

The Java-Updater/Helper-Tool Yin-Yang of Doom

Briefly, here are the basic mechanisms of the update-checking system as it is now (here’s the launchd manpage for reference):

There is a LaunchDaemon (com.oracle.java.Helper-Tool.plist) and a LaunchAgent (com.oracle.java.Java-Updater.plist) installed in the usual place within /Library.

They are actually symlinks to the web plugin’s installation area, in /Library/Internet Plug-Ins/JavaAppletPlugin.plugin/Contents/Resources.

The LaunchDaemon is not run on a schedule, but rather triggered by changes to the LaunchAgent plist file using a WatchPaths entry.

The LaunchDaemon runs Helper-Tool, a Bash script whose purpose is to enforce (based on default times set by an install script) or randomize the StartCalendarInterval key in the LaunchAgent plist, and reload the LaunchAgent. It modifies the file that triggers itself to run… to modify the file… that triggers it to run… you can probably see why this may not be a great idea.

The LaunchAgent runs the Java Updater binary located inside a .app bundle in the aforementioned Resources directory.

Java Updater.app is essentially a vessel for the Sparkle updater framework, which simply manually calls Sparkle’s “check in background” function. It is a pure Cocoa app. It does not seem to have any link to the Java plugin, settings configurable via the control panel, etc. It is not linked to any Java library.

There are a several major issues with this LaunchAgent/Daemon combo that I’ll cover in a future post.

For now, there are a couple possible workarounds I’d consider that are the least intrusive and most reliable way of suppressing the update check behavior. They each still have issues that are very important to understand if you want to deploy this and not dig yourself out of a hole later. I don’t consider either of them acceptable in the long term, and can’t recommend one over the other – it all depends on your environment.

Neuter Sparkle

One approach is to leverage the configurability of Sparkle. Because Java Updater is directly invoking Sparkle’s check method, overriding check behavior preference keys as outlined here won’t help, as far as I can tell. If you can at least read some Objective-C code you can look at Sparkle’s SUUpdater.m and see for yourself what it does, and make a pretty good guess at what methods Java Updater is calling by inspecting its binary strings (hint: the resetUpdateCycle and checkForUpdatesInBackground methods).

Sparkle supports overriding some preference keys that would typically be defined in a bundle’s Info.plist file, by setting them in the bundle’s user defaults domain instead (at either the user or system level). In this case, we’re looking at the com.oracle.java.JavaAppletPlugin domain, and the SUFeedURL string key. (It might be worth pointing out now that the Java 7 package uses about 4 different preference domains for the plugin and helpers, which doesn’t make figuring these out any easier).

Setting SUFeedURL to an invalid URL like “nil” (or one that doesn’t actually contain a Sparkle appcast feed) will cause Sparkle to fail silently. It’s worth noting, since this is being run as a LaunchAgent, that this doesn’t cause the actual Java Updater app to fail with a non-zero exit code. As far as Java Updater is concerned, it’s done its job and Sparkle just didn’t have anything for it to do.

There are alternate, more desperate Sparkle-related tweaks possible that will yield similar results to the above, which will be in a future post.

Advantages: This preference key can be managed like any other: a defaults command writing the value to /Library/Preferences/com.oracle.java.JavaAppletPlugin, MCX, or a Configuration Profile. The preference can later be removed/changed if desired, and it’s completely independent of the plugin installation.

Disadvantages: This overrides the update URL for anywhere the plugin may want to check for updates, meaning that if one checks the Control Panel manually for an update, it will not be able to check or verify whether it is up to date. It will state that it was last run whenever Java Updater was last run, but that it is “Unable to check for updates”, and to “Please check your internet connection and try again.” You’re essentially removing the ability for the plugin to update itself via its built-in mechanisms, even the user- or support-initiated ones. This puts it roughly on par with the state of deployingAdobe Flash.

Remove (but don’t disable) the LaunchAgent

A second approach is to prevent the LaunchAgent from ever running Java Updater in the first place.

After installing Java, unload the job temporarily, and remove the symlinks so that it’s not loaded again after a restart. You’d run something like this with elevated privileges:

You might think, like I did, that you can just launchctl unload -w the job to set it as permanently disabled. It turns out you absolutely should not do this, due to an oversight in the Java 7 installer’s postinstall script. Having these jobs disabled will cause the script, and thus the entire install, to fail.

Advantages: We don’t need to mess with any configuration of the updater mechanism itself.

Disadvantages: We’re still changing the “expected state” of the Java installation, and the fact that disabling the job permanently (ie. launchctl unload -w) causes future installations to fail doesn’t inspire confidence that any of Oracle’s future pre/postinstall scripts will be robust enough or perform even basic sanity checks. We also need to make sure that we unload the job and remove symlinks after every installation. Unloading is actually optional, since that the next update check will usually be a week from the time of installation, and a reboot will cause the launchd jobs to never be loaded again.

Current status

I don’t think either option is generally viable, especially if you’re in environment with a diverse set of client configurations (laptops, admin users, remote workers and sites, etc.) and need to make as few assumptions as possible about how client machines are used and maintained. If, on the other hand, you maintain control over at least software installations and updates, you might decide one of these two workarounds could work despite its limitations, at least until a better solution is discovered or implemented.

After I complained in ##osx-server on IRC, Michael Lynn found a channel through which to report bugs against the JRE, and Rich Trouton documented the steps and caveats in a blog post. If you support Java on your Macs and this is an issue for you, file a bug!

]]>http://macops.ca/everything-youll-wish-you-didnt-know-about-disabling-java-7-updates/feed/15New utility: XProtect Packagerhttp://macops.ca/new-utility-xprotect-packager/
http://macops.ca/new-utility-xprotect-packager/#commentsMon, 11 Feb 2013 17:13:58 +0000http://macops.ca/?p=368Roughly a week after the first widespread panic with the XProtect mechanism disabling Java on OS X, the same thing happened with the Flash plugin, with Apple issuing a definition update blocking all old versions about 3 hours after the latest Flash was available. (At least this time a newer version was available.)

It’s clear that a management strategy could be very useful in environments where users aren’t admins on their computers and can’t install updates themselves. One such strategy is to simply disable the updater, but the definitions should still be pushed to clients as you roll out new plugin versions, to enforce minimum security requirements as well as be able to protect against known malware.

There was some talk on Twitter, IRC and multipleposts on Greg Neagle’s blog. I dug around the XprotectUpdater binary and posted some ideas on how one could monitor this feed for changes.

I later realized that there are also multiple definition files: one for each major version of OS X, starting with Snow Leopard, when XProtect was first implemented. This means there will soon be four separate defintions to keep track of (at least until Apple stops providing security updates for Snow Leopard).

I wrote a basic utility to automate packaging up these changes as the definition file (‘clientConfiguration.plist’) is updated from Apple. Because what XprotectUpdater does to synthesize the two Xprotect definition files on the client is very simple, this tool can do the same thing, but for all available client versions. Meaning, you can run the command on a single machine (and on a schedule, if you wish) and automatically build new packages for all OS X client versions you support.

It’s also able to automatically push these updated packages to a Munki repository.

On the subject of Munki, there are a few different “auto-packager” tools that have been made available by different people. Because I wanted this tool to be self-contained and finished as quickly as possible, the mechanisms for building the package and importing into Munki are quite basic, and the tool’s functionality would probably be better served by integrating into a tool like Per Olofsson’s AutoPkg or anotherrecipe-based checking/building/importing tool for Munki integration. If one tool starts to see some more widespread adoption and community contribution, it would be great to integrate XProtect Packager’s functionality into it.

]]>http://macops.ca/new-utility-xprotect-packager/feed/1Monitoring Apple’s XProtect meta feed for changeshttp://macops.ca/monitoring-apples-xprotect-meta-feed-for-changes/
http://macops.ca/monitoring-apples-xprotect-meta-feed-for-changes/#commentsSat, 02 Feb 2013 20:22:30 +0000http://macops.ca/?p=342Greg had an interesting blog post yesterday on handling Apple’s XProtect Updater mechanism for managed environments, as admins were still scrambling to resolve clients that suddenly had their Java Web Plugin disabled and no newer version available to install that would satisfy Apple’s minimum version requirements defined in its XProtect blacklist (new versions of Java 6 from Apple for OS X 10.6 and Java 7 from Oracle have since been posted).

Maybe you’d like to at the very least know when this has been updated, and what are the nature of the changes. Here’s another example where the strings command proves useful, and it’s quickly obvious what’s going on.

If we look in /System/Library/LaunchDaemons for something related to XProtect, we find com.apple.xprotectupdater.plist. Opening it, we see it simply runs the executable at /usr/libexec/XProtectUpdater every 86400 seconds (or 24 hours).

Now, run the strings command on this binary, and see a few telltale methods and values (this is taken from somewhere in the middle):

Taking note of the URL at the top, we can simply curl this URL to see that it’s nothing more than a plaintext XML plist prepended with a security signature (the same signature that’s getting written to XProtect.meta.plist. Looking at the other strings, it’s clear that it’s using the Last-Modified HTTP header to compare this date with the current system date (“Last-modified date is later in time than current date”). It’s also using the Version key in the plist to determine whether the plist available from Apple is more recent than the one already installed.

Note, the “2” in the URL ..xprotect/2/clientConfiguration.plist is what was returned on my Lion machine. Snow Leopard clients look for “1”, Mountain Lion clients look for “3”, and so on.

Now we can throw this plist URL into a site like ChangeDetection.com, and ask it to check this URL once per day and send us an e-mail if it’s changed. We can see the details on the change, and when it was last modified. Update: Given that Apple’s updated this list within hours of new Flash/Java releases and will probably continue to do so, checking daily is probably not frequent enough.

We could also use an application like Jenkins, that already handles polling, jobs, and notifications. The URLTrigger works well for this, and we can simply ask it to track the last modified date of the URL however frequently we’d like, up to the minute if we so wish. From this point on we could write the XProtect.meta.plist file ourselves, package it and automatically push it to test clients, or anything we could dream up.

If we want to extract more information, we can also write a very simple script to do so, put it into a versioning system along with the PluginBlacklist information, etc. This example just prints out the version of the meta plist:

Lots of ways to solve a trivial problem! I’m not advocating whether or not to take measures to disable the XProtect mechanism on all your managed clients, but this might at least give some ideas on how you can at least be kept up to date with Apple’s new minimum security requirements for plugins, known malware, and anything else Apple will add to XProtect in the future.