Cocoa Therapy

Inside Macintosh

The “Inside Macintosh” book series, started in 1985, is without doubt a milestone in the modern computing experience, and not just on the Mac.

Inside Macintosh was largely a reference to what has today become Carbon. Part of the first volume was what has become the Human Interface Guidelines. The main goal of the HIG was to push applications towards a consistent appearance and behaviour. What is remarkable, and what still defines the Mac user experience today, is that Mac developers have pretty consistently adhered to the HIG.

The current Human Interface Guidelines is a three part document: “Application Design Fundamentals”, “The Macintosh Experience” and “The Aqua Interface”. The first two, a fifth of the content, largely pertain to the design process and common sense application design issues. The third part, over 80% of the content, is a very specific and detailed set of guidelines on the purpose, interaction and visuals of the elements of the Aqua UI.

In the 23 years since the first Inside Macintosh much has changed in the Mac look and feel, but I believe a turning point in the evolution of the HIG was the development of iLife around 2002-2003. Apple engineers and designers experiment outside of the sanctioned guidelines. Apple is continuously pushing the envelope beyond HIG conformance out of necessity, to innovate and refresh the look and feel, and iLife was and is the testbed for this.

Extending the Interface

In “Extending the Interface”, the guidelines leave the door open for extensions to the Aqua UI: “When a need arises that can’t be met by the standard elements, you can extend the set of controls […]”. So you’re violating the HIG only if you’re extending the interface when a standard element would meet the needs of the UI you’re implementing.

In the context of solving user needs, this explains why for example Aperture and Final Cut use grayscale controls and translucent (“HUD”) panels: standard controls and panels create visual noise that interferes with the perception of color of the actual content the application is trying to render. But does it also explain the iPhoto toolbar location? The iTunes 7 dark scrollbar look?

A reasonable argument might be that while Apple is technically violating the current HIG, they aren’t violating future guidelines: system-provided controls and frameworks, and the guidelines themselves, will eventually be updated to reflect the innovation.

What should a third party developer do? John Gruber’s C4[0] talk “Consistency vs. Uniformity in UI Design” spurred some debate and spread the “the HIG is dead!” meme.

A guideline is not a hard rule, and the HIG is open to extensions when “a need arises” anyway. I can’t help thinking that while there are important details regarding what Aqua controls should look like, they are still only details. If you look at any productivity app, you’ll find that the major part of the UI is composed of elements that are impossible to codify in guidelines.

The image editor view of Acorn, Pixelmator, Photoshop? The spreadsheet and charts editor in Numbers and Excel? The waveform view in FuzzMeasure, Fission or Logic? Clearly the visualization and the interaction depend on the nature of the data, and while there are several types of data that have natural or established visual mappings, interaction and editing are a whole other can of worms. This is where developers spend a lot of their coding time, and take the tough decisions that make or break the app.

An application with visually distinctive window style or control look will never stand a chance against an application with distinctive data visualization and interaction. As JLG once put it, “At a risk of being called sexist, ageist and French, if you put multimedia, a leather skirt and lipstick on a grandmother and take her to a nightclub, she’s still not going to get lucky” — he was referring to Windows 98 at the time, but it’s a timeless quote.

Visual user interfaces

Now imagine you’re John Geleynse (oh perhaps you are — hi John!), you desperately want to help Mac developers build better apps, you’ve covered the use of system-provided user interface elements, now what? The closest to describing what goes in the main content view is probably the “Reflect the User’s Mental Model” section in the “Human Interface Design Principles” chapter. Good, solid principles, but kinda fuzzy when you’re trying to build a new UI.

The Apple Human Interface Guidelines never covered the hardest part of building user interfaces, beyond general high level design principles. What’s changing now is what is a good versus merely acceptable user interface:

a table with numbers (covered by the guidelines) becomes a 3D shaded translucent chart (not covered)

a set of parameters controlled by sliders (covered) is now a rich visual representation of the parameter-generated data set (not covered)

a list of cities in a popup menu (covered) is now a map with the cities over it (not covered)

… and so on.

Many apps already expose core content data visually. For many of the apps that don’t, the future is for the standard controls-based UI to be replaced with application specific graphic design and interaction design, packaged in a custom view with contextual editing controls and direct manipulation.

Edward Tufte’s work on information visualization is a great source of inspiration, I believe the main view of many apps could be implemented as interactive Tufteian graphics. Bret Victor dissects this process in his amazing Magic Ink essay, where he describes how he built the user interface for the award-winning BART widget (pictured above).

In my experience Apple UI people are quite helpful in anything from brainstorming to refining UI concepts, for example in the WWDC labs and UI design sessions. However, discussions inevitably are on a case by case basis. It has become apparent that building a high quality app on the Mac now requires having a designer on the team.

Visceral user interfaces

How has GUI interaction evolved? Let’s put a square on screen:

You are prompted for X&Y coordinates, you enter them, click a button, the square appears;

You are shown a square, you tweak the X&Y numbers and it moves;

You are shown a square, you drag a horizontal and vertical slider and it moves;

You are shown a square, you click on the square and drag it where you want it;

You are shown a square, you barely touch the screen and the square follows your finger.

Each step removes a little bit of abstraction and a little bit of indirection, until interaction is natural, because while technically there is an interface, you’re manipulating the content with your hand and you don’t feel like there’s an intermediary.

Is it dead?

The HIG is still good. In fact the first fifth of it is pure gold, still 100% current and relevant. The Aqua guidelines are just less relevant to applications where user interaction and data representation are tightly coupled, where content and UI aren’t separated by indirection, where interaction is more visceral, where the content is the user interface.

Comments

[…] prime and needs to be updated or relocated to the trash bin. But, today Duncan Wilcox argues that The HIG is still good on his new blog, Cocoa Therapy. The HIG is still good. In fact the first fifth of it is pure […]

Nick, I don’t understand how Leopard’s GUI, which is the same in regards to scrollbars as Tiger disappoints you because of iTune’s scrollbars which are also the same in both Leopard and Tiger. I can only assume you are upset that nothing changed? That you hoped Leopard had adopted the iTunes scrollbars? I think it is iTunes that is inconsistent. However, I could reason that since Coverflow was integrated into the Leopard Finder, that if Leopard scrollbars looked like those in iTunes, that it might be harder to distinguish between iTunes and some Finder windows. So, that could be the reason they changed then in iTunes rather than in all of the OS.

That’s a wonderfully insightful comment, thank you JLG. Though in a way I guess it means I could have not written this post at all!

I’m still hoping what I wrote might instill doubt in developers coding a custom look for Aqua controls. Doing it without a reason is not only pointless, it also takes development time away from possible interaction improvements.

While iTunes’ scroll bars don’t look like the standard Aqua components, they certainly don’t behave like they are either.

For users of graphics tablets and Inkwell (like me!), iTunes has a problem where the scroll bars don’t actually behave correctly when a pen is the mouse—if a user presses (clicks) on a scrollbar, Inkwell is supposed to be able to tell that the user wants to click immediately on the control rather than wait for electronic ink input; instead, Inkwell actually waits for electronic ink input over the scroll bar first, which causes headaches when you actually want to scroll a list of songs in iTunes!

Inkwell is supposed to be sensitive to where the pen is pointing in order to decide whether to enter electronic ink mode or mouse mode—the menu bar, the window title bar, the scroll bars and the grow (resize) box are examples of standard Aqua user interface elements where mouse mode automatically overrides electronic ink mode in Inkwell. However, in iTunes, the scroll bars are custom controls, and they don’t allow Inkwell to work the way the standard controls do, causing problems relating to how users can control iTunes with a pen as the mouse. Specifically, users are forced to wait for the double-click time (as set in Mouse Preferences) before Inkwell realises that you want to switch from electronic ink mode to mouse mode in order to move the scroll bar.

Another reason why iTunes shouldn’t get away with using its scroll bars!

What the human interface guidelines still convey effectively is that a developer should approach building a UI with a deep respect for the user. What metaphors the user is familiar with, what mindset the user is in when using the app, what visual affordances work best.

As a developer you want to make the user feel at home, get the user in a state of flow, in “the zone”, get the user productive and not interrupt her with a confusing UI or stupid error messages.

Whoever builds weird UI skins for the sake of different isn’t really doing the user any service. Honestly I don’t know if my post is confusing the matters more, as I see many comments nitpicking about consistency. Consistency for standard aqua controls is a moot point. Duh.

What I’m trying to show is that to give the user a better interaction, more flow, etc. user interfaces are forced to migrate from aqua controls to custom views. Look at the BART widget, the user interface there is no longer an aqua table of aqua cells, it’s a work of graphic design, and it’s not a mythical new interaction model. It undoubtedly creates a much better user experience. Would you argue it shouldn’t be done for the sake of consistency?

New comment

Recent posts

About the Author

Hi, I’m Duncan Wilcox, I’m a software developer and chocolate addict, living in Florence, Italy. I’m passionate about the Mac, photography and user interaction, among other things. Contact me at duncan@wilcox.it or follow me on Twitter. These days I work on Sparkle.