Apple augments reality, tells you where to go

Apple has filed a patent application for putting a mash-up of augmented reality and turn-by-turn navigation onto your iPhone.

The filing, entitled "Augmented reality maps", describes an enhancement of how iOS's currently combines the capabilities of the iPhone's location services–enabled Maps app with the device's internal magnetometer in order to display route information and show you which direction you're facing on that route.

Apple's filing adds to that exisiting capability an augmented reality component. The user would point the iPhone's video camera at his or her surroundings, the device's location and directional sensors would tell a remote or local database where it was, and the user could query that database for directions to a destination or class of destinations.

The database would then overlay the video stream with talk-ballons indicating possible choices. The user would select one, and augmented reality–style turn-by-turn navigation would then be overlain on the video stream during the trip, with the directional data coming either from the device's memory or from the online source.

In this San Francisco example, two parks are suggested as destinations (and, yes, Dolores does have palm trees)

Althought the filing doesn't mention it, one would hope that voice-over guidance would also be provided.

The device's location would be determined by GPS, cell towers, or both; its tilt angle by its accelerometer; and the direction it's pointing by its magnetometer – and we can only hope that Apple equips its iDevices with a more-accurate magnetometer before it implements the ideas explored in this patent filing.

The iPhone's Maps app works with the magnetometer to show what direction you're facing (right) – if it works (left)

The iPhone's accelerometer would also be part of the sensor mix, not only to indicate whether the user was facing uphill or down, but also to change the display to something more useful than the user's feet, should he or she be holding it more or less horizonally.

The system could suggest only locations that are within the angle of view of the device's camera, and to display only those locations in the augmented-reality view. As you aim the device in other directions, other items would pop up in the augmented-reality view.

As is the case in most patents of this type, a wide variety of other "embodiments" are also discussed. For example, the system could provide augmented-reality displays of location information and directions that would be provided not in response to a user's query, but instead in response to images of locations it recognizes in the video stream, or simply from location coordinates provided by the device's sensors.

In those latter two cases, the opportunities for paid advertising are obvious. ®