Facebook Augmented Reality

Recently I tried my hands at REAL Facebook app. I had previously made a little app, just to test the API and Web Science wanted to run a test case to prove they could do it, and to help sell apps to clients.

What does the app do?

The app gives you the opportunity to create a custom newspaper cover story featuring one of your friends (or yourself). The next step allows you to print out an Augmented Reality (A.R.) marker which you can then use to literally hold the custom newspaper in your hands. You can then snap a photo of you with the paper in your hands and post that picture on your friends wall. A photo album is also created to hold all of the images you create with the app.

How does the app work?

The structure of the app basically consists of 3 parts. Customizing a newspaper cover, posing with your newspaper, and finally, posting your photo to your friends.

Part 1 – Customizing the newspaper – If you have tried the tutorials on this blog you will know how to get permissions and get information about the user. With this information in hand I created a simple form with an image of the newspaper on the right. The only part of the form that uses Facebook user info is the drop down with all the friends names. This is a simple way to choose a friend and while I did experiment with some Auto-Complete tutorials and source code I found it too complicated (and buggy) for the task.

Using the Graph Api, I can now use that friend’s ID to get their profile image and put it, along with a title and text, into the newspaper template. For all of the image manipulation tasks I used an open source PHP library called WideImage. I use this to put white squares over the placeholder text on the paper, to position and resize the profile photo and to eventually save that image to our server.

Wide Image is great and so easy to use. I did have to research a bit deeper to create a multiline textbox, however, and found a solution by a guy called SK89Q.
With the newspaper fully prepared, I pass the location of the image to the next page where Flash was handling the Augmented Reality part.

Part 2 – Augmented Reality snapshot – Using the FLARToolKit combined with Papervision (the standard setup) we take the image from part 1, apply it to flat 3D plane object and orient that object according to the AR marker being read by the webcam. When the user snaps a photo of themselves, the image data is converted to base64 text and sent to the next page as a variable On the following page the image can be decoded and saved using PHP’s imagecreatefromstring().

Part 3 – Posting the image – The very first prototype had all the photos being hosted from our work server. We later decided that this would be a big drain if the app became popular and decided to create an album for our app’s photos. The album is great as it creates a permantent location for all of the photos where they can be commented and liked, furthering their viral capability. This was actually quite easy and only required a bit of extra work. We needed to add photos and albums to the permissions that the app asks at the start. We also put put a link back to the app in the description of the album. Along with an album, the photo is also posted directly to the chosen friend’s page using CURL.

What problems did we encounter

Well the first major problem was our own server, which was not running the right version of PHP for the Facebook php class. To keep the pace going I developed on my own webspace for a while.

There are also some funny bugs that people should be aware of regarding Internet Explorer. Explorer was blocking cookies in an iFrame when the top page was on a different domain. After much searching I found the solution was to set a particular header with PHP .

Another Explorer problem I had was image caching. Because the user can use the preview function to continually change the same image, IE was always presenting the same cached image from the first time it loaded. I fixed this by applying a random number variable to the end of the image location.

What would we like to improve

At the moment the 3D newspaper turns out quite pixellated and I am not sure why. The image that I pass to the flash is good resolution, so I think it will be a matter of playing with the 3D settings and maybe looking into another Flash 3D solution (UPDATE: It was just a matter of setting a smooth parameter on the bitmapTexture object).
I also worry that the app may become too slow when there is a load on the server as WideImage is not known to be fast. To fix this I may need to rebuild using PHP’s native image manipulation classes which are more difficult to use but should be faster.

And when I have time I want to have language selection at least for english.