Description

Today's project show's off one of my favorite new capabilities available with Windows Phone 7.1(5)*, how you can incorporate augmented reality into your "real world" applications. We've seen the demo's and samples, but today we're looking at an AR toolkit that adds AR to your geography based apps.

Image you're near a "restaurant row" trying to decide which one to go to. Sure you could look up each place but where's the cool that? Instead, what if you were to simply hold up your phone, point it at the restaurant and then see floating over it its rating? Then as you turn around, as each restaurant comes into view its rating also appears. Now that's cool...

And that's what we're highlighting today... An app that does that and the toolkit that makes it possible.

GART was created to help people quickly and easily build Augmented Reality applications for Windows Phone.

This kit is different from other AR kits in that it enables what we call “Geo AR”. Where other toolkits place virtual things on top of specially printed tags, this toolkit places information on top of real places in the world around you. It does this by tracking where you are and the direction you’re facing.

Geo AR apps are easy to write because all you need to provide is a collection of objects that have Latitudes and Longitudes. These can come from anywhere, for example a Bing restaurant search, a Flickr photo search or a Wikipedia article search. The framework then takes care of managing sensors and tracking where the user is in relation to the reference points. It can show where the points are in relation to user from a top-down perspective (on a map) or it can show where the points are as a virtual lens into the real world.

Please note that GART makes heavy use of the Motion APIs shipping with Windows Phone Mango (OS 7.5). You will need a motion-enabled device to use GART. This should include all devices that ship with 7.5 as well as many of the existing 7.0 devices that have been upgraded to 7.5. The emulator, unfortunately, does not currently support Motion APIs. GART will fail gracefully if motion is not supported by raising the ARDisplay.ServiceError event.

Here's a snap of the Solution;

And just a bit of code;

private void searchService_SearchCompleted(object sender, SearchCompletedEventArgs e)
{
// Only proceed if the search was a success
if ((e.Error == null) && (e.Result.ResponseSummary.StatusCode == ResponseStatusCode.Success))
{
ObservableCollection<ARItem> items = new ObservableCollection<ARItem>();
// Clear out existing items
ARDisplay.ARItems.Clear();
// Add new results in
foreach (SearchResultBase result in e.Result.ResultSets[0].Results)
{
// See if this search result is a business result
BusinessSearchResult businessResult = result as BusinessSearchResult;
// If it's a business result, convert it to a restaurant item and add it
if (businessResult != null)
{
// Create the new restaurant item
RestaurantItem ri = new RestaurantItem()
{
Cuisine = BingDataHelper.GetCuisineName(businessResult),
GeoLocation = BingDataHelper.GetGeoCoordinate(businessResult),
Name = businessResult.Name,
Rating = businessResult.UserRating,
};
// Add to the collection
items.Add(ri);
}
}
ARDisplay.ARItems = items;
}
}

So now you're excited and ready to add GART to your project? Make sure you check out the documentation.

Samples included in the toolkit

The Geo AR Toolkit contains two samples that show how to build an app from scratch:

SimpleAR – Simply adds labels around you in random locations. Look for "add locations" and "clear locations" in the application menu.

BingAR – Uses Bing to display information about restaurants nearby. Displays the restaurant name, the type of food they serve and their average user rating.

Creating an app with the toolkit

The basic steps anyone would follow to build an app with the toolkit are:

Start a new Windows Phone project

Add an ARDisplay control to your page

Add the views (or layers) you want as children of the ARDisplay

In Page.OnNavigatedTo call ARDisplay.StartServices()

In Page.OnNavigatedFrom call ARDisplay.StopServices()

Create a collection of ARItem objects (or your own custom type that inherits from ARItem)

Set the GeoLocation property of each ARItem to a location in the real world

Set ARDisplay.Items equal to your new collection

(Optional) Style visual elements to give them a custom appearance or show custom data

How the toolkit works:

ARDisplay is thework horse of the solution. When StartServices is called, ARDisplay takes care of creating AR services, initializing them and subscribing to their events. When StopServices is called, ARDisplay takes care of unsubscribing from the services and freeing their resources. When we say ‘AR Services’ we're talking about the PhotoCamera (for the background video feed), Location for Lat / Long positioning and Motion for Heading / Attitude. Each of these services can also be individually Enabled or Disabled for applications that don’t need or want them. For example, you may want to set LocationEnabled to false and set the Location property to a known location while you’re developing at home.

The next part of the solution are the individual views (or layers) that can be added as children to ARDisplay.

Currently there are four:

...

If you're interested in Augmented Reality/AR, geography or camera based app's GART might just be a great place for you to start looking...

* Due to the Windows Phone Mango version numbering "fun," whereas its internal version, and the version associated with the SDK, is Windows Phone 7.1, but it is being marketed as Windows Phone 7.5, when referring to Mango, I'm going to be using "7.1(5)" until someone yells at me... lol