Windows Phone : Build an Advanced Camera App for Nokia Lumia Phones

In this article, I’m going to teach you how to develop an app for the 41-megapixel (MP) Nokia Lumia 1020 and 20MP Nokia Lumia 1520 smartphones. I’ll focus primarily on the Nokia Lumia 1020, but the information applies to all Lumia Windows Phone 8 devices with PureView technology. First, I’ll discuss PureView, the technology behind the powerful camera included in the phones, and then I’ll explain the advanced features available to access and enhance your photographs. I’ll provide an overview of the Nokia Imaging SDK (it has tons of ready-to-use graphic filters) and a walk-through of how to use it. I’ll also cover the typical workflow needed to build a camera app and show you how to create a tilt-shift photo filter to simulate shallow depth of field. Let’s get started.

Understanding the PureView Technology in a 41MP Camera Phone

PureView technology consists of advanced camera hardware and related software. Together, they allow the capturing and saving of high-quality, high-resolution images. The three main aspects of PureView technology are a high-resolution camera lens, oversampling and lossless zoom. I’ll briefly explain each one.

The core of PureView technology is a high-resolution sensor, 7,728 pixels wide and 5,368 pixels high, totaling more than 41MP. This enables the camera phone to capture big photographs, six times larger than a normal 5MP photograph.

Figure 1 compares a 41MP resolution picture with a 5MP resolution picture. Because of this 41MP resolution, you can take high-quality 34MP 16:9 photographs (7,728 x 4,354) as well as 38MP 4:3 photographs (7,152 x 5,368), as shown in the lens view of the camera in Figure 2.

Figure 1 A 41MP Picture Compared with a 5MP Picture

Figure 2 Lossless Zoom Is Still Part of the Photograph

Pixel oversampling is when the camera takes a 41MP photograph and creates a high-quality 5MP image. This is the picture you see on the phone’s screen. Furthermore, the oversampling process keeps all the rich detail in the image but filters away any visual noise.

The third aspect of PureView technology is lossless zoom, which simply means you don’t lose image quality when you zoom. This is possible because—at any point in time—you have available the original 41MP picture, which is oversampled to show a 5MP photo. When you zoom, you’re really zooming into a part of the original 41MP photo. At up to a 3x zoom, you’re still dealing with parts of the original 41MP photo. And even at maximum zoom, your photo quality is still the quality of a regular 5MP photo. Due to oversampling, the picture quality only gets better as you zoom. Figure 2 shows how the lossless zoom is still part of the original photograph.

The Nokia Imaging SDK

When building a camera app, a high-resolution photograph is the raw material, but you’ll need an advanced software stack that can use the huge images and let you access and manipulate them. This is where the Nokia Imaging SDK comes into the picture (bit.ly/1hJkmpl). It provides a set of advanced features to access a high-resolution photograph taken by the camera.

The Nokia Imaging SDK includes partial JPEG decoding, called Random Access JPEG (RAJPEG) technology. RAJPEG allows for random access of JPEG data, fast downscaling of images and instant partial decoding. These aspects of RAJPEG enable real-time image manipulation. The Nokia Imaging SDK contains more than 50 filters, effects and enhancements. It even contains a set of the most common image operations, such as crop, resize, rotate and undo, just to name a few. These out-of-the-box features help you create advanced camera/photo-centric applications without worrying about the most common functionalities. To use the Nokia Imaging SDK in your project, follow the instructions on the Nokia Lumia Developer Library page (bit.ly/KzDPNG).

Imaging SDK APIs

The APIs available in the Imaging SDK can be easily applied to the image data captured by the camera in relatively few lines:

Here, e.ChosenPhoto is used to create a StreamImageSource. The e.ChosenPhoto can come directly from a camera-captured photo or from any photo in the album. I then created a filter effect to apply on the photo and added an artistic FogFilter to the array of filters in my effect. If desired, this array can contain multiple filters, which are then applied and rendered to myImageBitmap using a WriteableBitmapRenderer. As the name suggests, it renders an image source to a WriteableBitmap. Finally, the RenderAsync method helps in the asynchronous rendering of the image.

Along with FogFilter, you can apply more than 50 other Imaging SDK filters to a photograph. Figure 3 shows the list of enhancement filters inside the free App Filter Explorer, available out of the box for developers. It includes filters such as AutoEnhanceFilter, AutoLevelsFilter, ColorBoostFilter and ExposureFilter. These are filters you can directly apply to any photo or real-time captured image to enhance quality.

Figure 3 Enhancement Filters in the Free App Filter Explorer

There are other filters for brightness (BrightnessFilter) and contrast (ContrastFilter), which take parameters for brightness and contrast. There are a number of filters just for artistic effects, such as emboss, grayscale and sepia. There are also the image-editing “filters” that provide the previously mentioned commonly used functions such as rotate, flip, mirror and so on. These can be applied to a single image multiple times, one on top of another, to create dramatic effects.

High-Resolution Camera App Workflow

Now that I’ve told you about the hardware and Imaging SDK capabilities, I’ll show you the typical workflow you need for building a powerful camera phone app. Because I’m dealing with a huge image, I have to play nice so my app doesn’t make other apps slow or take a huge amount of memory. This can be ensured by not trying to directly manipulate the big image in the camera.

At any point of time, the camera will have two photographs: the original 41MP photograph and the 5MP photograph, which is shown in the camera view panel. In my app, I’ll always work on the 5MP camera phone picture, which can be either the oversampled image of the whole photo, the zoomed part of the picture, or the oversampled and partially zoomed part of the photo.

Figure 4 shows the workflow for the imaging app.

Figure 4 Image App Workflow

The individual steps include:

Capture a high-resolution image.

Save it to your local storage.

Do your magic using the Imaging SDK.

Scale down the resulting image to 5MP.

Save the 5MP-enhanced image to the camera roll.

Creating a Photographic Filter: Tilt Shift

Now I’ll walk through the workflow of my photographic filter-based, tilt-shift camera app. Tilt-shift photography is the use of camera movements—specifically tilt—for selective focus to simulate a miniature scene (see Figure 5). You can find more information on tilt-shift photography at bit.ly/1bRYYNK. In this app, I’ll simulate the shallow depth of field with digital post-processing using multiple filters in the Nokia Imaging SDK.

Figure 5 The Tilt-Shift Effect

I’ll show you how this fits in the workflow I discussed earlier.

Acquire Photo The first step for any camera-based app is to acquire an image either from the phone’s camera or image gallery. You can do this in three different ways. One is using the default 5MP camera by using the CameraCaptureTask, which triggers the normal camera viewfinder. The second method is by using the native photo picker control called by PhotoChooserTask. These two alternatives are sufficient for most camera apps, but for an advanced app where you need to capture high-resolution photos, you need to create a custom viewfinder, which triggers the built-in Pro Camera application viewfinder with high resolution. This is done using the object PhotoCaptureDevice. See Figure 6 for the list of methods and properties supported by PhotoCaptureDevice.

Figure 6 Methods and Properties Supported by PhotoCaptureDevice

The UI The tilt-shift app consists of two images: one Original­Image to show the actual image captured or selected using the photo chooser, and the TiltShiftImage, which displays the final image after the tilt-shift filter is applied to the original image. The SelectButton triggers the native photo chooser, and the Camera­Button triggers the camera, as shown in the code in Figure 7.

Capture a High-Resolution Image The first two methods of normal camera and photo picker functionality are straightforward. The Microsoft.Phone.Tasks namespace has two task objects, Camera­CaptureTask and PhotoChooserTask, for these two purposes. Simply select the image either from the photo chooser or from the result of the camera capture as the source of your tilt-shift filter:

Do the Magic First I’ll use the PickImageCallback to get a photo, which is then set as a source of the OriginalImage. To access dimensions of the image, I’ll use ImageProviderInfo.

I’ll also create filters and apply them in the session. To create a tilt-shift effect, I’ll use three different filters: BlurFilter with KernelSize 15; ColorBoostFilter with value 0.5; and again BlurFilter with KernelSize 21. The two blur filters make the foreground and the background of the image out of focus and the ColorBoostFilter brightens the region, which I want to do to create the miniature effect. The code for this is shown in Figure 12.

To apply the tilt-shift filter, I need three rectangles: the top rectangle for blur; the middle rectangle for color boost; and the bottom rectangle for blur again (see Figure 13). In this example, I’ll use a rectangular region called TiltshiftRegion, which the user can touch and move to customize the position where the tilt shift takes place. The translucent rectangle shown in Figure 13 becomes tilt shifted and the rest of the area is blurred.

Figure 13 The Three Rectangles Used for the Tilt-Shift Effect

The TiltShiftRegion position is used to calculate the three rectangles where the filters are applied (see Figure 14).

And I’m done. You now understand PureView advanced camera capture and post processing of the image using the Nokia Imaging SDK. I hope you found this article useful, and I look forward to hearing your comments and suggestions.

Rajesh Lalworks at Nokia and is passionate about Windows Phone and Web technologies. His latest book is “Digital Design Essentials: 100 Ways to Create Better Desktop, Web and Mobile Interfaces” (Rockport Publishers, 2013). You can find it at bit.ly/dsgnbk. For information about the author, check out iRajLal.com.

Thanks to the following technical experts for reviewing this article: Chevon Christie (Microsoft) and Paras Wadehra (Nokia)