A blog for developers programming with Autodesk platforms, particularly AutoCAD and Forge. With a special focus on AR/VR and IoT.

February 11, 2013

Using SLAM-based Augmented Reality to visualize 3D geometry

I first became aware of the work being done by 13th Lab a couple of years ago, but just last week someone pinged me about it again and re-triggered my interest (thanks, Jim :-).

13th Lab is a small Swedish company that has created some really interesting Augmented Reality technology. Many AR systems make use of fiduciary markers (which often look like sections of QR codes) to make it easier to determine where the 3D content should be positioned and visualized in the 2D image of the scene being fed from your device’s camera.

I’m far from being an expert in AR, but I thought I’d have a play around with some technology that 13th Lab have recently released into Beta (both of which are “FREE (even for commercial use)”). They’ve created two products: an SDK (currently for iOS and Unity 3D – apparently Android support is on its way) – which I decided not to spend time on – and a browser you run on an iOS device that loads custom web-pages.

The PointCloud Browser can be installed from the App Store and pointed at one of the samples that show you how to use the JavaScript APIs to do something simple such as add basic 3D primitives or go much further and implement an AR-based game. Using one of these samples as a base, I created my own web-page that pulls down 3D fractal data from the web-service I implemented as part of last year’s the Cloud & Mobile series (if you try to load this page in a standard browser you’ll get uninteresting results, by the way).

Once loaded, you can point your camera at a flat surface with some helpful detail (SLAM systems are presumably greatly helped by the existence of unique visual detail). I chose the whiteboard in our kitchen:

When you tap the screen, the browser then uses computer vision techniques to capture information about the surface:

At which point you should first see your 3D geometry:

As you move the device around, you see difference views on the geometry, of course. Here’s a close up:

And here’s a view from the other side of the whiteboard:

The results aren’t yet quite as I want them, even if they prove the concept: the spheres currently all have the same radius, for instance, as that gets set at the mesh level (and mesh objects are ideally shared by multiple nodes). Some work will be needed either to find a way to set this per instance or to establish an efficient way of generating mesh objects for unique size/colour combinations.

Here’s the HTML and JavaScript code that integrates this data from the web-service:

If you want to give this a try yourself, install the PointCloud Browser and enter “http://autode.sk/appar” in the address bar: this will re-direct to the longer URL on my blog.

Once I’ve found a way to adjust the radii per instance, I think I’m going to investigate exporting simple 3D geometry from AutoCAD, to see what’s possible (the browser does support .OBJ for more complex objects – I’ll be looking at fairly simple stuff).