ARKit Tutorial: How to Make an Augmented Reality App For iPhone and iPad

October 3, 2017 5479

Who should read this blog:

Entrepreneurs – Who wants to get the basic understanding of how AR apps for iPhone are developed with an example.

iOS App Developers – Who wants to learn how to make an Augmented reality app from basics.

There are so many apps on the Apple App Store that it has become increasingly difficult to develop a mobile app that sets your business apart from others and from your competitors. Today, we don’t need another calendar app or to-do-list app, but still, we see many apps popping up in the App Store regularly.

It’s basically a messaging app like whatsapp, but it consists of well-defined features and specific targeting to set itself apart from the other messaging apps for iphone.

This is the first lesson to take away from this blog. Focus on your target audience with an extreme precision and build your app features around them.

Augmented Reality (AR) Is a Technology, Not a Feature

Many times clients come to us and ask how they can integrate AR in their existing mobile app. Sometimes they just want to put up a new button in their app that starts some sort of Augmented Reality experience for their users.

But you might have heard the saying that goes like “Don’t put the cart before the horse”, which means don’t use Augmented reality just for the sake of using Augmented reality.

That approach was already possible few years ago, when the technology was fresh, and you could just wow users by floating a 3D model in a room.

Nowadays, that formula does not work. People expect better AR experiences. And up till now, only a few companies were able to provide them. IKEA for example. IKEA built its AR app with Apple’s ARKit to let their customers preview furniture before making the purchase.

Not just IKEA, but Apple’s new ARKit has helped many businesses to bring augmented reality experiences for their customers. One of the recent examples is Topology Eyewear. It is an augmented reality app-based startup providing custom-fit glasses from a 3D scan of your face.

Topology just launched in July and says it has already had orders in the “low hundreds.” It’s now starting up a digital marketing campaign to increase those orders and awareness of the product and has taken in about $4 million in initial seed capital from the founder of OPI nail polish, George Schaeffer, to help it grow.

Make Augmented Reality The Main Component of Your App’s Goals

As we said that showing a floating table or chair just doesn’t do the trick anymore. So, to successfully leverage the full potential of ARKit in your app, It has to accomplish a major problem of your target audience.

Are you selling a complex product that is hard to grasp in a video or in images with descriptive text alone?

If the answer to this question is YES, then you should create an augmented reality app for your business by hiring reliable iPhone app development company.

Now, teaching how to build a complete, feature-rich Augmented reality iOS app is beyond the scope of a single blog. However, we can show you the basic steps to make an augmented reality app for iPhone. Our Apple ARKit tutorial will display an AR dragon breathing fire to the next object.

Let’s Get Started!

Overview

Augmented reality (AR) describes user experiences that add 2D or 3D elements to the live view from a device’s camera in a way that makes those elements appear to inhabit the real world. ARKit combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.

Basics

SceneKit / SpriteKit knowledge

Basic knowledge of Matrix formulas and transformations

Before diving into 3D models user should know about basics of Matrix formulas. Let’s see how Matrix works.

Matrix

A matrix (plural matrices) is a rectangular array[1] of numbers, symbols, or expressions, arranged in rows and columns.[2][3] For example, the dimensions of the matrix below are 2 × 3 (read “two by three”), because there are two rows and three columns:

A matrix is a rectangular array of numbers or other mathematical objects for which operations such as addition and multiplication are defined.[6] More general types of entries are discussed below. For instance, this is a real matrix:

The size of a matrix is defined by the number of rows and columns that it contains. A matrix with m rows and n columns is called an m × n matrix or m-by-n matrix, while m and n are called its dimensions. For example, the matrix A above is a 3 × 2 matrix.

Matrices which have a single row are called row vectors, and those which have a single column are called column vectors. A matrix which has the same number of rows and columns is called a square matrix. A matrix with an infinite number of rows or columns (or both) is called an infinite matrix. In some contexts, such as computer algebra programs, it is useful to consider a matrix with no rows or no columns, called an empty matrix.

Beginning with ARKit

When using ARKit, you have three options to create your AR world:

SceneKit, to render 3D overlay content

SpriteKit, to render 2D overlay content

Metal, to build your own view for an AR experience

In this augmented reality app tutorial, we’re going to explore the basics of ARKit and SceneKit by building a simple demo application.

At the end of the demo, You can play with virtual Dragon. Looks promising right? So let’s get started.

Let’s start by reviewing what you’ll need to develop and run this project.

Install the Software You Will Need

Xcode 9 and iOS 11:

ARKit requires an iOS device with an A9 or later processor for a full AR experience. So it will only work on the following devices:

iPhone SE

iPhone 6s

iPhone 6s Plus

iPhone 7

iPhone 7 Plus

iPad Pro (All three variants and models)

New 9.7-inch iPad (2017)

Creating the project

Open Xcode 9 and create an AR app (New project).

Enter the project information, choosing Swift as the language and SceneKit as the content technology and create the project.

AR cannot be tested on an iOS simulator, so we’ll need to test on a real device.

When the app is run, you’ll be asked to give permissions to the camera.

After that, a new model will be added to the scene when you run the application and positioned according to the orientation of the camera.

Now that we have set up the project, let’s take a look at the code.

How SceneKit works with ARKit

ARSCNView

The ARSCNView class provides the easiest way to create augmented reality experiences that blend virtual 3D content with a device camera view of the real world. When you run the view’s provided ARSession object:

The view automatically renders the live video feed from the device camera as the scene background.

The world coordinate system of the view’s SceneKit scene directly responds to the AR world coordinate system established by the session configuration.

The view automatically moves its SceneKit camera to match the real-world movement of the device.

If you open Main.storyboard, you’ll see there’s an ARSCNView that fills the entire screen:

This view renders the live video feed from the device camera as the scene background, placing 3D images (as Scenekit nodes) in the 3D space (as ARAnchor objects).

When you move the device, the view automatically rotates and scales the images (ScenKit nodes) corresponding to anchors (ARAnchor objects) so that they appear to track the real world seen by the camera.

This view is managed by the class ViewController.swift. First, in the viewDidLoad method, it turns on some debug properties of the view and then creates the SceneKit scene from the automatically created 3D scene:

ARSession

ARSession is heart of the ARKit. AR session manages motion tracking and camera image processing for the view’s contents.

Then, the method viewWillAppear configures the session with the class ARWorldTrackingConfiguration. The session (an ARSession object) manages the motion tracking and image processing required to create an AR experience:

You can configure the session with the ARWorldTrackingConfiguration class to track the device’s movement with six degrees of freedom (6DOF). The three rotation axes:

Roll, the rotation on the X-axis

Pitch, the rotation on the Y-axis

Yaw, the rotation on the Z-axis

And three translation:

Surging, moving forward and backward on the X-axis

Swaying, moving left and right on the Y-axis

Heaving, moving up and down on the Z-axis

An ARAnchor uses a 4×4 matrix represents the combined position, rotation or orientation, and scale of an object in three-dimensional space.

In the 3D programming world, matrices are used to represent graphical transformations like translation, scaling, rotation, and projection. Through matrix multiplication, multiple transformations can be concatenated into a single transformation matrix.

If W == 1, then the vector (x, y, z, 1) is a position in space. If W == 0, then the vector (x, y, z, 0) is a direction.

Now that you understand how the sample works, let’s modify it to make our demo.

Integrate custom .scn model to the application. Like below:

You can change your 3D model name and check demo application.

How to integrate the custom model into the project:

So the default template project comes with a 3D model which we need to replace with the one we just created. Looking through our Project Navigator, find a folder called art.scnassets. This is the spot we are looking for. Default application comes with default ship.scn model. First, delete ship.scn model from the application.

Add your .scn model to the application. (Here we use dragon.scn model).

Find dragon assets from source.

Note: You can also use .dae model to ARScnView.

Click on dragon.scn file, see how it looks.

Now select device and run the application, you will find the animated dragon on camera.

Conclusion

With this blog, we’ve seen how to create an augmented reality app for iPhone and iPad using ARKit. However, if you’ve any ideas on augmented reality projects, and want to hire ARKit developers for it, don’t hesitate to contact us.

In case, if you still have any query or confusion related to iPhone app tutorial, then you can get in touch with us through our contact us form. One of our sales repersentatives will revert to you within 48 hours. The consulation is absolutely free of cost.

Author Bio

Designation: iOS Team Lead

Hitesh Trivedi is an iOS Team Lead at Space-O Technologies. He has over 9 years of experience in iOS app development. He has guided to develop over 100 iPhone apps with unique features and functionalities. He has special expertise in Swift and Objective-C.