A/V Production – iOS to macOS

Console – Gaining Insight into Audio & MIDI behaviors on iOS & macOS

Console is a useful app for producers to take a deep dive into the behaviors of the audio & MIDI systems of both macOS and iOS. Here we’ll look into how to make use of Console.

Start by opening up Console from the Utilities menu of the Applications folder. As soon as you open Console you’ll notice a flood of messages. Console monitors all the traffic on your Mac so it’s fairly overwhelming when thousands of messages are likely flying by on your screen.

The first step is to narrow down these messages to the ones we care about, and to customize the categories of messages displayed. At the top of the monitoring window where all the messages are flying by, you’ll notice some categories like Category & Message. You can right click on that toolbar to see additional message types to display, and then drag them around to reorder as you see fit. You can also do the same via the View menu.

The next recommendation is to utilize the Search window. Type in MIDI for example and Console will only display messages where ‘MIDI’ appears. Of course, it’s challenging to know which messages are applicable to our needs, but by starting with some simple searches it’s much easier to then further refine our searches. After performing a search, a Save option appears, allowing us to recall our search parameters from the menu above the messages window.

Once you’ve typed in a search term, you’ll notice a drop down menu appears to the left of your search term. The options here can further refine your search. You’ll notice all these search refining terms are the same that can be displayed on the bar above the console window. Even if you don’t know what they mean, by performing a few searches you’ll start to get an idea of the different message types, and which correlate to audio & MIDI.

You can continue to refine your search results by searching and categorizing multiple message types, and again saving them for recall later.

Above I’ve entered a search for the Subsystem com.apple.coremidi. I discovered this type of message by doing a basic search for MIDI. As messages appear, you can then select them, learn a bit more about them, and if they seem relevant – include them in your search.

To monitor your iOS device, plug it out to your Mac via USB. Once you do the device name will appear in the Devices list to the left of the Messages window. Select your iOS device, and Console will be monitoring messages from that device.

Initially for me, pruning down the amount of messages was the easiest way to get going with Console. Within a minute or two of connecting your iOS device to Console, it’s likely that you’ll have a list of 100K messages in your Console window, even if you’ve no apps running on your device. Apparently there’s a lot going on behind the scenes!

So now let’s try to prune down those messages, and at the same time we’ll want to get some MIDI or audio functionality going on our device so we’ll have something relevant to monitor. This is where it gets fun, as you’ll gain some useful insight into how things work.

Try a search for MIDI – hopefully there aren’t many, or any messages appearing in the Console window – if there are, you’ve got some MIDI-related activities going on! After entering a search term – you might want to click on the Clear button above the Console window to remove all the existing messages.

Next up, open an app on your iOS device that has some MIDI functionality. I’m going to open AudioBus 3 because it’s an app I use everyday, and I know it has some MIDI functionality. Even with our MIDI search Filter – I still got 670 messages that almost instantly appear when I opened AudioBus 3. Below you’ll see my Console window, with the MIDI search filter in the search window, and the 1st message received when opening AudioBus 3 highlighted in green. I’ve customized the the view to include categories like Process, Subsystem, Message, Category, PID, and Library.

In the image above, I’ve highlighted another message from my search – this message let’s me know that the MIDIServer is filtering MIDI-CI messages – which are MIDI Capability Inquiry messages, part of the new capabilities of the MIDI Standard, and already included in both macOS and iOS – hopefully these will be incorporated by developers soon! You can see from the image there is a window below the Console messages window that provides additional details about whatever message you have selected. You can then set up search filters by PID, or Thread ID for example to greatly refine the results of a search. Of course, it’s helpful if you have some idea what you are looking for, but I’ve been quite surprised at the amount and variety of messages being passed. As a general example, if you switch an iOS app to the background there are a bunch of messages. Any sort of change to the audio session requires updating all the participants, endpoints, and the like. The audio system is impressive, and complex, monitoring messages with Console gave me a respect for the audio system, and more sympathy for apps utilizing core audio and MIDI.

I’m hoping to next create some search filters to test sample rates, aggregate and multi-output audio devices. and the like. My understanding is that depending on device, and the state of the audio session, that sample rates might be opaque to us end users, in that apps can only set a Preferred sample rate. I’d also like to get a grasp on measurement mode, and the Sound Check setting in iTunes. I’m also hopeful to get some insight into the clock selection process in macOS for aggregate audio devices, and how drift mode works. Console also provides options to output a timestamp for every message, which could be helpful in measuring updates to the audio session, or the order in which endpoints are updated. And of course, interrupts and prioritization of messages can also be helpful in shaping an audio session.

Some Examples:

Type in IOAudioControl to the search window. Adjust the volume a bit on your output device, and you should see a bunch of messages fly by. In the image below I adjusted the volume slider of my system volume from mute (0). I’ve highlighted three sections in Green that showing ascending values from 1266 up to 2323. In between those sections are varying length messages with a 0x0. So I’d surmise that volume adjustments, when detected are measured 8 times per frame, and return a 0x0 intermittently where pauses are detected. Another message on this thread lets me know the integer range of the output volume is 0 to 9472. Probably with some further sleuthing I could figure out how the integer value correlates to the volume slider, my guess would be 64 bit.

Switching to iOS, and then changing the volume by a single press of the physical volume button on my iPad, I captured a few messages with a search for ‘audio.’ The messages show a volume change of delta 0.062500 for the Audio/Video route. then the AVSystemController updates the telephony subsystem of the volume change as well. Leaving the window open a bit, the image below shows as well the Gradmaster Clock of the audio system updates itself once a minute. Latency, Route Changes, Sample Rates, and other highly detailed messages can be captured with Console. Both for troubleshooting, developing, and sculpting audio sessions Console is a handy resource built-in to the Mac’s Utilities.