UI Browser — Features

The ultimate assistant for Accessibility and GUI Scripting

Scroll &varr;

PFiddlesoft has unmatched experience exploring, monitoring and controlling the graphical user interface of applications running on Macintosh computers. Our products carry on the tradition of PreFab Player, a seminal product released in 1994 for the classic Mac OS. Now, since 2003, PFiddlesoft brings our expertise to you on macOS with UI Browser, a utility to support Apple's Accessibility and GUI Scripting technologies.

GUI Scripting is a technology for macOS having many similarities to PreFab Player. It allows you to manage most Macintosh applications by writing simple AppleScript scripts that automate its windows, menus, buttons and other User Interface elements, even if the target application is not scriptable on its own.

UI Browser is a user interface navigator

With UI Browser, you can explore almost every window, menu, button and other UI element of most macOS applications. You can see at a glance, in a familiar macOS browser view, where all of them are in the application's containment hierarchy, easily navigating through windows, sheets, drawers, dialogs and other views. To keep you from getting lost, UI Browser's Path view gives you a compact outline of the path from the target application's root element to any UI element you select, with an ordered list of all intermediate elements, including some that are not normally visible to users. UI Browser can even highlight the selected UI element on the screen to help you identify it, and follow the current focus as you use the target application. You can also see in UI Browser's Attributes drawer dozens of properties of any UI Element in the target application, including its role, title, position, and size, and its current state.

UI Browser is a user interface observer

You can tell UI Browser to monitor notifications that are broadcast by macOS applications when any changes take place in their user interface—whether because a user clicked a control in the target application, chose a menu item, or typed some characters, because an AppleScript command took effect or a network administrator or user took some action, or because you did something to the target with UI Browser itself! You can use notifications to monitor an application's response to GUI Scripting and other commands.

UI Browser is a user interface actor

You can tell UI Browser to control macOS applications by setting the value of user-settable attributes in the target application's UI elements—the size and position of windows, whether the application is frontmost or hidden, whether a window is focused or minimized, whether an outline or table row is selected, and much more. You can also perform every action supported by the target application's UI elements, such as clicking its menu items and buttons and confirming text field entries. You can even send keyboard shortcuts to the focused element of the target application and enter individual characters into the active text field or text view of the target application.

UI Browser and Accessibility

UI Browser and Apple's GUI Scripting both rely on Apple's Accessibility technology, which is designed to make the Macintosh accessible to persons with disabilities. This technology was new to the Macintosh in 2002, in Mac OS X 10.2 (Jaguar), and Apple has made substantial improvements since then.

UI Browser and GUI Scripting

GUI Scripting is a macOS AppleScript technology that uses simple, plain-English commands to explore, monitor and control applications by scripting their user interface. Because it is based on Apple's Accessibility API, which is built into every standard User Interface element in macOS , it allows scripts to work with applications even if they are not otherwise scriptable. UI Browser brings previously impossible automation tasks within reach of every scripter.

Apple introduced full support for GUI Scripting based on Accessibility in Mac OS X 10.3 (Panther) in October 2003. GUI Scripting was originally made available the year before, in Mac OS X 10.2.3 (Jaguar), through a public beta version of Apple's System Events application. Because System Events has been installed by default on every Macintosh computer since Mac OS X 10.3 (Panther), you can distribute your scripts for use "out of the box" without requiring any special supporting software.

GUI Scripting brings Apple's Accessibility technology to every scripter. It doesn't matter whether the target application was written for the Cocoa environment or the Carbon environment, or whether it was written in C, C++, Objective-C or Swift. If it was written using one of Apple's standard programming environments, Apple has already made sure it is accessible. As Apple continues to improve Accessibility in the future, UI Browser will automatically take advantage of the changes.

There is one aspect of GUI Scripting that makes scripts difficult to write: how to figure out the Accessibility containment hierarchy of the target application's UI elements. Every application has its own idiosyncratic arrangement of containers, many of which are internal coding conveniences and not visible to users. Furthermore, many UI elements have no names and therefore have to be scripted by their index numbers—which often have no relationship to their visual layout. While you can use AppleScript itself to decipher these mysteries, it is very awkward and time consuming.

UI Browser saves the day! Just fire it up, choose the target application or use the built-in Screen Reader or press a hot key to read the screen under the mouse, and look at the browser view in the main window. After a few clicks and a couple of scrolls, you see the exact Accessibility hierarchy, complete with every UI element's title (if it has one) and its AppleScript name and index number.

We even save you the trouble of writing it down. With UI Browser's unique AppleScript menu, several different kinds of AppleScript statement are generated for you, including object references, get and set commands for object properties, action commands, and keystroke commands. The generated AppleScript statement appears in a separate window. If you aren't sure whether you prefer the AppleScript name reference form or the index reference form when both are available, click a button to switch instantly between them. The generated AppleScript statement is already selected for you, ready to edit, or to copy and paste or drag into your favorite script editor. Better yet, set a preference to copy it to the clipboard or send it directly to your default script editor automatically.

It doesn't end there. Before you design your script, and while you're fine tuning and testing it, use UI Browser's ability to set attributes and send actions to make sure the target application will respond as you expect. Not every application is fully accessible, in part because some applications include custom user controls that haven't yet been made accessible. Instead of banging your head against the wall for hours because you think you haven't written your script correctly, just try to set the property or send the action using UI Browser. If it doesn't work in UI Browser, it most likely isn't your script that is at fault. Or, just maybe, UI Browser will help you find a way to make the target application respond that wasn't immediately obvious to your AppleScript instincts. UI Browser is not written using AppleScript or GUI Scripting; instead, it is based directly on the same Accessibility API that Apple's System Events application uses to implement GUI Scripting. If UI Browser can't make the target application dance, GUI Scripting probably can't either.