Archive for the 'Augmented Reality' Category

I’ve been busy writing lots of words over the last couple of months. I had my first magazine article published a couple of weeks ago. It was for Web Designer magazine and was on, yep you guessed it, augmented reality 🙂

It’s an investigation into AR in terms of the current state of play and where we are headed. I also discuss my approach, and that of others to the technology as a platform for art and storytelling. It’s on the shelves now.

I also wrote a guest post for the Creators Project called Augmenting The Future Of Art. This is focused more on how artists are using AR and the importance of investigating the boundaries and social implications of the technology.

Thanks to Mark Billen (WD) and Kevin Holmes (CP) for the opportunity to contribute to their respective publications.

In the coming years, once Augmented Reality glasses are commonplace, I hope to see apps that allow for creative user generated content to be distributed in virtual space. These tools would give people the opportunity to virtually build sculptures, paint on walls and leave trails using gesture and voice as they wander through cities. Others plugged into the same AR channel would see these digital artefacts seamlessly attached to the places they were created. Apps such as TagDis have explored this area, but, as with all mobile AR, there is a sizable barrier in the act of navigating to and using an app on a handheld device. Ubiquitous AR experienced via a head mounted display could transform whole cities into virtual art galleries.

Konstruct explores this vision by investigating generative art in an Augmented Reality environment. It is a sound reactive AR experience for the iPhone that allows the user to create a virtual sculpture by speaking, whistling or blowing into the device’s microphone. A variety of 3D shapes, colour palettes and settings can be combined to build an endless collection of structures. Compositions can be saved to the device’s image gallery.

Konstruct is a free app available on iPhone 3GS and 4 running iOS 4+. A version for the iPad 2 is planned in the coming months.

In terms of the credits, I came up with the concept and developed the app. Juliet Lall was responsible for designing the user interface and all branding including the Konstruct website.

Users are encouraged to email virtual sculptures to konstruct[at]augmatic.co.uk to be featured in the Flickr gallery.

Vodpod videos no longer available.

So on to the technology. I used the new iPhone tracking library String. I’ve been experimenting with the beta version of String for a few months now and it’s surprisingly powerful – I managed to fit over 100,000 polys into a scene without any noticeable dip in frame rate. The level of accuracy in the tracking is also very sturdy. I’m told that that this has improved considerably with the latest iteration of the beta SDK so I’ll be updating the app very soon.

There are currently 2 approaches to developing String apps, Unity, and straight up OpenGL ES. With the Unity option, developers can be up and running within minutes. Being a sucker for punishment I decided to take the OpenGL ES and Objective-C route. This was my first proper experience using OpenGL. Coming from a Flash background, where most of the hard work is wrapped up for you with libraries such as Away3D and PaperVision3D, it was a bit of a learning curve. Ultimately though, it was a great experience to learn about and implement buffer objects, matrices, normals etc. I look forward to learning more about OpenGL and GLSL for future projects.

OK lots of things to mention so I thought I’d be lazy and lump them into one blog post.

I’ve spent the last month beta testing a new iPhone tracking technology called String. I’ll leave the details for a future blog post, but I will say that it is one of, if not the fastest and most accurate mobile trackers I have seen to date. Also, importantly, there is no license fee for artists and designers wanting to experiment with and publish apps using the tech. It’s about to be released any day now. Definitely one to watch.

I’m currently experimenting with generative art in AR space using OpenGL ES. Should the app be accepted into the store, it will be included in the Square Art exhibition. The theme is Blank Canvas, which ties in freakishly well with what I have in mind.

I just had an article published on the Young Creative Council (AKA YCC London) blog. Every week they get a guest author to write about whatever they like. I used it as an opportunity to write about my work, my inspirations and my thoughts on AR in terms of the current state of play, the future and my approach to it as a platform for art. I don’t tend to use my blog as an outlet for those sorts of ramblings so it was nice to write down my thoughts on these matters for a new audience.

You can read the article here. Big thanks to Laura and Emma for sorting this out.

I’ve been asked to both speak at and be on the advisory board for the AR Summit. This event is penned to be the biggest augmented reality conference in the UK and is taking place in London in June. If you don’t fancy the presentations, there will be an exhibition hall showcasing the latest and greatest in the world of AR. Definitely worth a visit.

It’s a huge honour to be given this opportunity. I’m looking forward to having a say in the organisation of the event. Hopefully I can inject a creative edge into the proceedings.

I was featured in this month’s Web Designer magazine. The article is titled “Whatever happened to Augmented Reality?”. I attempt to answer this question and give my thoughts on current trends, mobile AR and the future. I’ve been informed that I also had an interview published in the iPhone special of Computer Arts Projects about my iPhone app Fracture last month. I didn’t actually manage to get a copy so if anyone has the issue and a few spare minutes I’d really appreciate a scan.

Hologramatic paper, advanced finger tracking and GPS based graffiti writing in Silicon Valley. These were just a few of the highlights from the superb ARE2010 conference. I was lucky enough to be invited to talk at this flagship event in Santa Clara (see my previous post for info on my session) and can honestly report that I have been informed, inspired, rendered speechless, entertained and (surprisingly) well catered for.

The names on the keynote list were impressive to say the least. Science fiction writer and prophet of the augmented reality industry Bruce Sterling, and gaming legends Will Wright and Jesse Schell provided insightful, inspirational, encouraging and hilarious ponderings on AR. In approaching the subject they employed a range of disciplines such as anthropology, philosophy, sociology, ethics, biology and many others. Describing these talks would involve a whole other blog post. Do yourself a huge favour and check them out for yourself:

Personally, I was looking forward to seeing and experiencing the new technology on offer and between the exhibitions, the talks and the Auggies (the AR demo competition) there was certainly a great deal on show.

As to be expected, there were many mobile AR demos. e23games showcased their impressive new iPhone application TagDis which allows users to virtually graffiti their surroundings. You can choose from a variety of icons, stencils or use their gesture based system to tag up your neighbourhood. More info here.

Another excellent and slightly curious iPhone app was one presented at the Auggies by Occipital. The app hasn’t been named yet but it made use of the new camera API available in OS4 and appeared to record the camera feed in the centre of the screen and ‘fill in’ the blank areas surrounding it as the presenter pointed and moved it across the bewildered audience. Very clever indeed. I for one will certainly be downloading this once it is available.

T-Immersion, One of the biggest names in AR, showcased a few of their current projects which utilise their R&D endeavours. One such upcoming project was their Shrek campaign which involved background removal and advanced face tracking. Another interesting demo made use of finger tracking, an image was seamlessly manoeuvred around the screen by the turning and pinching of a hand. Another finger tracking example allowed the user to play a join the dots game. The virtually drawn line appeared to stick to the paper. Another double take moment. Bruno Uzzan, the CEO of T-Immersion also gave an interesting talk on standards in AR and suggested a standardised AR logo to be used in all products that utilise the technology. More info here.

Another AR giant, Metaio, also showed off some very impressive developments. They are currently pushing their mobile natural feature tracking system Junaio Glue, another technology that harnesses the new iPhone camera API to attach CGI elements to the user’s surroundings. They demoed a new mobile zombie shooter game for Android in which you blast or chainsaw your way through the virtually present undead. In another demo, Peter Meier, the CEO of Metaio attached a hovering dragon to the main stage whilst jumping about in his AR Adidas trainers. Quite a moment.

Art is a big passion of mine and I was excited to see a selection of talks entitled ‘AR in Art’ which also manifested itself as a series of installations that were set up in the Art Gala area.

Eric Gradman of the Syyn labs collective discussed his face/marker tracking application Cloud Mirror. This intrusive installation requires the participant to enter their Facebook details. Once these details are secured, their account is scoured for information which appears around them as speech bubbles. In addition to this, Twitter, IMDB and even the sex offenders register is scoured for further potential dirt.

Helen Papagiannis was next up to show her wares. She had an impressive body of marker based AR artwork to showcase including AR Wonder Turner, an exquisite corpse inspired installation which was also set up in the Art Gala Other works of note where AR Joiner Waterfall, Amazing Cinemagician, Magic tunnel popup book and the AR Popup Dollhouse.

Another artist that stood out was Fernando Nabais. Fernando is involved in performance based projection mapping which is a form of art that often takes my breath away. Many examples of this can be found at my Vimeo channel, Tech Art. He showed an example of a gorgeous text based piece he worked on which can be viewed below.

One of the best sessions I attended was offered by Christopher Stapleton of The University of Central Florida and Hitlab’s Mark Billinghurst and was on experiential learning. Christopher’s talk was a discussion of the importance of using Augmented and Mixed realities in learning environments with a big focus on perception, memory and imagination. He spoke with an enormous amount of energy and enthusiasm. I would definitely recommend reading some of his publications on this subject. Mark’s talk focused more on the many AR projects he has worked on in this area. I was particularly interested by the children’s AR workshops he had organised such as the Storytelling workshop and the BBC AR Jam. Mark demonstrated one of the tools he helped develop for this sort of workshop called BuildAR. Using this software he built an AR scene in about 2 minutes.

If there was a cutest application award it surely would have gone to Do crew by Whistlebox. This beautifully illustrated children’s site was one of the few Flash based AR applications on show and interestingly there was not a fiducial marker in sight. It is a series of fun motion capture based games that encourage kids to remove themselves from their desks and play using gestures. Surely a big hit with parents. I hope so for Whistlebox’s sake as this is a paid application! For those computer vision types reading this, I’m sure I saw Memo Akten’s MSAFluid library (most probably Eugene Zatepyakin’s port) used here.

Absolutely the most immersive AR experience I encountered at the event was the Touching Augmented Reality demo by YDreams. They were using a Project Natal-like camera by Canesta which could essentially detect the depth of people and objects and build a 3D model using this information. Combining this model with a collision detection algorithm allows the user to interact with 3 seperate modules in a surprising realistic manner; a collection of falling cubes can be caught, rebounded or swept off the (actual) table on which they land; a floating 3D metaball can be shaped and disrupted; a 3D camera can be seamlessly rotated in space. All in all, one of the finest and most intuitive interactive experiences I have ever had.

We were fortunate enough to receive a talk from Microsoft’s computer vision guru Blaise Aguera y Arcas, the architect of the incredible Photosynth and Bing Maps. In addition to several demos and walkthroughs, he gave us all an overview of how he was able to build a 3D point cloud from 2D images. I was blown away with the street level navigation in Bing maps, especially with the Photosynth integration.

One of the most stunning things I saw wasn’t even an AR application. Whurley (AKA the Evil Genius), the co-founder and CTO at Chaotic Moon (who was also helping organise the event) had brought along several sheets of hologramatic paper he had procured from Zebra Imaging that depicted urban scenes. This stuff totally blew my mind. It was the result of overhead drones taking some sort of ultrasonic pictures of cities. Absolutely mental. And it was great to finally meet Whurley after several email conversations. Below is a link to the technology he was exhibiting.

So all in all, ARE2010 was a big success. Ori Inbar, Tish Shute, Whurley and all the other organisers did a fantastic job in putting this together. It was so successful that ARE2011 was announced at the end of the event.

It was an honour to meet so many of the people I have admired and/or spoken with online in the last year since this crazy AR journey began. There were a great deal of friendly, enthusiastic, like minded people who where all very excited about shaping the future of human computer interaction. It was great to be a part of the event and share that enthusiasm.

I’m proud to announce that I’ll be speaking at The Augmented Reality Event alongside some of the biggest names in AR. It’s taking place June 2-3, 2010 at The Santa Clara Convention Center, CA, and It’s one of the first events completely dedicated to Augmented Reality:

You can receive the reduced $195 registration price – $200 off the $395 normal price – by using Discount Code: E195 during registration. This fee includes both conference days, lunch, reception and more.

To get a sense of the richness and depth of content, take a look at the ARE2010 Schedule – it will be THE must-attend AR event of the year: 2 days, 3 tracks, 40 AR companies; with keynotes:

This event will be a unique opportunity to see the “state of play” of the emerging augmented reality industry. Special events include:

– A press conference for new product announcements by leading AR companies
– Launch pad for new Augmented Reality start ups – a judging panel with VCs and entrepreneurs
– Over 80+ AR speakers over two intense days
– “The Auggies” – competition for the best augmented reality demo
– Art and magic live performances by AR artists

I was approached by the Adobe Platform Evangelist Tomas Krcha a couple of weeks ago with regards to showcasing a few of my AR applcations at a number of Adobe CS5 launches across Europe. He’s currently in the process of visiting Prague, Vienna, Bratislava, Athinai, Ljubljana, Berlin, London and Moscow to spread the word.

He sent me this video of the Prague event last week which put a huge smile on my face. Three applications are shown but the money shot is definitely at 1:15.

Big thanks to Tomas for reaching out. It was definitely a very proud moment and a huge compliment.