April 12, 2011

Apple Working on Simplifying Multi-Touch Gesturing

A new series of patent applications that have surfaced at the US Patent and Trademark Office recently, clearly indicate that Apple is working on new ways to simplify their multi-touch gesturing on both iOS devices and Mac's via multi-touch and/or Magic Trackpads. One of Apple's key patents within this group of nine, introduces us to some newly contemplated methods of gesturing. The methods include the use of mid-drag gestures, microgestures within gestures, and other gesture modification motions to modify and/or alter user interface behavior. One of the gesture modification motions already reviewed to date relates to Apple's use of Spaces on iOS devices. In the big picture, Apple states that their newly proposed methods and interfaces will reduce the cognitive burden on a user while saving battery life. If we're lucky, we may even get to see some of these new gestures playing out in various demo's at this year's WWDC keynote. Time will tell.

The Problem Apple's Patent is to Solve

The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch-sensitive surfaces include touch pads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on a display.

In these devices, the need for rapid object manipulations, mode changes, and simple programmatic input to modify or alter user interface behavior is critical. In some instances, users benefit from being able to alter their input gesture on-the-fly or in real-time.

But existing methods for real-time user interface input gesture alterations and modifications are cumbersome and inefficient. For example, using a non-contiguous sequence of gesture inputs, with at least one gesture to serve as a behavior modifier, is tedious and creates a significant cognitive burden on a user. In addition, existing methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices.

Accordingly, there is a need for computing devices with faster, more efficient methods and interfaces for modifying or altering user interface behavior. Such methods and interfaces may complement or replace conventional methods for modifying or altering user interface behavior. Such methods and interfaces reduce the cognitive burden on a user and produce a more efficient human-machine interface. For battery-operated computing devices, such methods and interfaces conserve power and increase the time between battery charges.

New Mid-Drag Gestures and Microgestures

Apple's patent introduces us to mid-drag gestures, microgestures within gestures, and other gesture modification motions performed contiguously within an overall gesture (i.e., without losing contact with the touch-sensitive input surface during the gesture) to provide intuitive ways to interact with a user interface for varying purposes, such as modifying user interface behaviors, changing optionally displayed items, etc.

Apple states that the use of mid-drag gestures reduces the cognitive burden on a user, thereby creating a more efficient human-machine interface. For battery-operated computing devices, enabling a user to use mid-drag gestures allows for faster and more efficient use of user interfaces, thereby conserving power and increasing the time between battery charges.

Applies to Document, Graphic, Gaming & Medical Applications

Various mid-drag gestures, microgestures within gestures, and other gesture modification motions performed contiguously within an overall gesture, may be used for any suitable purpose, including without limitation, turning on or off alignment guides, snapping to varying proportional display modes, changing anchor points in a document, using a microgesture as a substitute for a function key on a keyboard, snap-to-grid display mode, adding arrowheads or other features to displayed objects, snapping to various rotation angles, adding control points to curves, while dragging a figure over an electronic canvas, inserting one or more displayed objects at the current contact point in response to detecting one or more microgestures, transitioning a device to a next operational mode in a series of two or more operational modes, (e.g., transitioning through text-to-speech and displayed output modes, setting ring tones, setting auto-answer of a mobile phone, changing graphics modes etc.).

Games: according to Apple, the new gestures will be able to be used for controlling games, (e.g., shifting gears up or down with a micro-gesture while steering with a two-finger rotational gesture, transitioning through a list of weapons).

Medical: medical staff will be able to use a zoom control while shifting through images (e.g., switching image modes between various medical imaging modalities like MRI, fluoroscopy, CT scans, PET scans, etc., without interrupting their zoom-level).

New Gestures Presented

Below you will find a major chart containing three sets of newly proposed gestures based on using one, two and three finger combinations. Many of these gestures have been given descriptive names such as "backtrack" and "star" to assist users in remembering their movement configuration in an easier manner.

Apple's patent FIG. 10 is a set of exemplary illustrations of two-finger microgestures in accordance with some embodiments. FIG. 10a illustrates a "radial tick" with a first finger contact microgesture; FIG. 10b illustrates a "radial tick" with a second finger contact microgesture; FIG. 10c illustrates a "radial tick" with both finger contacts microgesture; FIG. 10d illustrates a "tangential tick" with a first finger contact microgesture; FIG. 10e illustrates a "tangential tick" with a second finger contact microgesture; and FIG. 10f illustrates a "tangential tick" with both finger contacts microgesture.

Three Finger Gestures

Apple's patent FIG. 11 is a set of exemplary illustrations of three-finger microgestures in accordance with some embodiments.FIG. 11a illustrates an "axial tick" with a first finger contact microgesture; FIG. 11b illustrates an "axial tick" with a second finger contact microgesture; FIG. 11c illustrates an "axial tick" with a third finger contact microgesture; FIG. 11d illustrates an "off-axial tick" with a first finger contact microgesture; FIG. 11e illustrates an "off-axial tick" with a second finger contact microgesture; FIG. 11f illustrates an "off-axial tick" with a third finger contact microgesture; FIG. 11g illustrates a circular microgesture with a first finger contact; FIG. 11h illustrates a circular microgesture with a second finger contact; and FIG. 11i illustrates a circular microgesture with a third finger contact.

In the short term I'm going to have to take Apple's word on their new methods being less complicated, because after viewing hundreds of patent illustrations related to this patent series, it sure looks complicated to me. Then again, I'm certainly no "power user" when it comes to using my multi-touch devices. I'm still at the goo-goo gaga level. I doubt that I'll be first in line to sling an "arrow" or "radical tick" on my iOS devices. But if you are a power user, then you're going to be able to rock'n roll even faster with yet more advanced and …oh ya … simpler new gestures.

If you want to get drowned in the details of this massive Apple patent, then check out patent application 20110074830. Apple credits their UI Design Manager Christopher Weeldreyer, Peter Rapp, Akiva Leffert, Jason Mar and Jay Capela as the inventors of this patent applicationwhich was originally filed in Q3 2009. A total of nine patents were generated by Apple's engineering team regarding new gesturing advances that are now in the works. It should be noted that Apple's name doesn't have to initially appear on patent applications until they've been granted. This method allows the patent to go undetected when simple searches are made for "Apple" patents. This is why Patently Apple hyperlinked to Weeldreyer's Linkedin profile above so that we could prove to you that in fact you're looking at an Apple patent.

Notice:Patently Apple presents only a brief summary of patents with associated graphic(s) for journalistic news purposes as each such patent application is revealed by the U.S. Patent & Trade Office. Readers are cautioned that the full text of any patent application should be read in its entirety for further details. Patents shouldn't be digested as rumors or fast-tracked according to rumor time tables. Apple patents represent true research that could lead to future products and should be understood in that light. About Comments:Patently Apple reserves the right to post, dismiss or edit comments.

Note: The sites that are linked to above offer Apple community members with an avenue to make comments about this report in many original languages. Additionally, many of these sites provide our guests with different takes on any given patent or concept that is presented in our reports to make it more fun, interesting and/or personal. If you have the time, join in!