I’m very happy to announce my new book, Learning AV Foundation, is available for pre-order from InformIT. Enter MCCUNE1808 to receive a 35% discount. This is the book I wish I had a few years ago, and I hope it will provide you with the understanding and inspiration to build amazing media applications for iOS and OS X!

This book is not intended to be a definitive reference guide covering every aspect of the framework, but instead focuses on the most relevant parts of the framework to lay the foundation that will empower you to be fully comfortable with the concepts, features, and conventions used throughout. Here are the details of the topics covered in the book:

Chapter 1, ”Getting Started with AV Foundation”—This chapter will help you take your first steps with AV Foundation. It deconstructs the framework to help you gain a better understanding of its features and capabilities. This chapter also provides a high-level overview of the media domain itself and covers topics such as digital sampling and media compression. An understanding of these topics will be helpful throughout the book.

Chapter 2, “Playing and Recording Audio”—AV Foundation’s classes for playing and recording audio are some of its most widely used features. In this chapter we discuss how to use the framework’s audio classes, and you’ll put them into action building an audio looper and voice memo applications. We also cover how to use audio sessions to help you provide a polished audio user experience to your apps.

Chapter 3, “Working with Assets and Metadata”—Much of the framework is built around the notion of assets. An asset represents a media resource, such as a QuickTime movie or an MP3 audio file. You learn to use assets and how to use the framework’s metadata features by building a metadata editing application.

Chapter 4, “Playing Video”—Playing video is one of the most essential tasks AV Foundation performs. It’s a primary or supporting use case in many media apps. You gain a detailed understanding of how to use the framework’s playback features to build a custom video player with full transport controls, subtitle display, and Airplay support.

Chapter 5, “Using the AV Kit Framework”—AV Kit is a new framework introduced in Mac OS X 10.9 and now in iOS 8. It enables you to quickly build AV Foundation video players with user interfaces matching QuickTime Player on OS X and the Videos app on iOS. This can be a great option if you want to build players maintaining fidelity with the native operating system while retaining the full power of working directly with AV Foundation’s video APIs covered in Chapter 4.

Chapter 6, “Capturing Media”—This chapter provides an introduction to the framework’s audio and video capture features. You learn to use these features to control the built-in camera hardware available on iOS devices and modern Macs. This is one of the most widely used areas of the framework, and it can help you build powerful, modern camera capture applications.

Chapter 7, “Using Advanced Capture Features”—This chapter covers a variety of advanced capture topics. You learn to use metadata capture to perform barcode scanning and face detection. You learn to use the advanced zooming capabilities provided by the framework. You also learn to enable high frame rate capture, which is great for adding slow motion effects to your videos. We also discuss how to integrate with OpenGL ES to process the video samples captured by the camera, which opens up a world of possibilities.

Chapter 8, “Reading and Writing Media”—AV Foundation provides a lot of high-level functionality, but the framework never hides the lower-level details from you when you need it. In this chapter we discuss the framework’s low-level reading and writing facilities that can enable you to process the media in any way you want. We discuss how to read audio samples from an asset and render them as an audio waveform. We also look at applying real-time video effects using the camera capture APIs.

Chapter 9, “Composing and Editing Media”—In this chapter we begin our exploration of the framework’s media editing features. This is one of the most powerful features of the framework, and it enables you to create new media by composing and editing media from a variety of sources. In this chapter you begin building the book’s most advanced application, 15 Seconds, which is a video editor similar to an application like iMovie for iOS.

Chapter 10, “Mixing Audio”—An important part of building media compositions is learning how to mix multiple audio tracks. You learn how to use mixing techniques such as audio fades and ducking that will help you add polish to your audio presentation.

Chapter 11, “Performing Video Transitions”—Video transitions are commonly used to indicate a change in location or storyline, and AV Foundation provides robust support for applying video transitions to your compositions. In this chapter you learn to use the framework’s video composition to control the compositing of multiple video tracks in your composition. You’ll put these features into action to add dissolve, push, and wipe transitions to the 15 Seconds app.

Chapter 12, “Layering Animated Content”—This chapter discusses how to add titles, lower thirds, and other animated overlay effects using the Core Animation framework. You’ll see how to use Core Animation to build animation sequences that seamlessly synchronize with your video playback. We also discuss how to incorporate these same effects in your final exported videos.

I mentioned earlier in the the year that a major focus of mine has been to help better educate people about AV Foundation. I’m happy to announce I’m working on a new book called Learning AV Foundation that will provide a comprehensive guide to mastering the framework. The book provides a hands-on guide to working with the various features of the framework and focuses heavily on building real-world applications for iOS and OS X.

The book is still early in its development, but the publisher has made a Rough Cut version available on Safari Books Online. The first 5 chapters are currently available and new ones will be added as the content is developed. A Rough Cut chapter is the raw, unedited version, so if you notice any errors or omissions, ping me here or leave a comment on the book’s page on Safari Books Online.

I had the great opportunity this spring to attend and speak at a couple different stops on the CocoaConf Spring Tour. I was at the event in D.C. in March and this past weekend returned from the event in San Jose. Now that I've had a couple CocoaConfs under my belt I thought I'd share my thoughts on the conference.

CocoaConf is a technical conference for iOS and Mac developers. The conference offers three concurrent tracks on a wide variety of topics so at any given time you're sure to find something of interest to you. The sessions offer a nice mix of introductory and advanced technical topics along with some great business-oriented talks as well. It has a really strong speaker line up featuring guys like Daniel Steinberg, Bill Dudney, Chris Adamson, and Jeff LaMarche, to name but a few. It was great to meet so many of the people whose writing and other works have been a big source of professional inspiration over the years.

One aspect of the conference I really like is the organizers put a cap on the number of attendees at a given event. Having a smaller group of attendees makes it much easier to meet new people, make new friends, and have meaningful conversations. I have learned a lot from the people I have met over the past couple months and listened to a lot of great stories along the way.

No mention of CocoaConf would be complete without discussing the family behind the conference. The Klein family puts on the conferences and does a great job of juggling the logistics while managing to make everything run like clockwork. It has been great getting to know Dave and his family a bit better. I can't say enough good things about the Kleins and am very thankful for their contributions to the Cocoa community.

The bottom line is CocoaConf is a great place to learn, connect, and grow. The CocoaConf Spring Tour may be over, but the bus will be rolling again this fall. Be sure to keep an eye on their website for upcoming dates. If you haven't been to CocoaConf before you owe it to yourself to go.

This past weekend I spoke at the CocoaConf event in DC. This was my first time attending CocoaConf so didn’t know what to expect, but I was very impressed by the format and quality of the conference and thought the organizers did an excellent job. I’m speaking again in San Jose next month and will post a more thorough review of the conference when I get back.

I gave a talk entitled “Composing and Editing Media with AV Foundation” where I dove into the details of the media creation and editing APIs. This is one of the more interesting areas of AV Foundation, but it is also the least documented which makes it particularly difficult to learn. Hopefully, this session and its associated sample app will make it easier for others to get started.

Tonight I gave an updated version of my “Learning AV Foundation” talk at the MN CocoaHeads meeting. In a rare move I actually posted my slides and sample code before I gave the talk. You can find the materials here:

AV Foundation is Apple’s advanced Objective-C framework for working with timed media in iOS apps. The framework, as we know it today, was first introduced in iOS 4 and has seen signficant additions and enhancements in both iOS 5 and iOS 6. Starting with Mac OS 10.7 it is also the default media framework for the Mac platform which means Apple is investing the bulk of its media engineering resources in its development. Clearly AV Foundation is important to Apple and if media matters to you and your applications, it should be important to you as well.

AV Foundation is a very impressive framework with a broad and powerful feature set. However, it’s not a particularly easy framework to learn to use. It’s quite large with many classes, functions, and protocols so simply understanding where to start can be a challenge. It also relies on a number of advanced language features such as blocks, KVO, and Grand Central Dispatch so it can be a lot to bite off if you’re just starting out with the Mac or iOS platforms. But the single biggest reason this is a difficult framework to learn is it is so poorly documented. The how to find what dating sites someone is on is badly out of date and the API docs can, at best, be described as sparse. My best guess as to why this is the case is that the development team’s technical writers are always on vacation at release time. This means the only real way to learn to use the framework is through a significant investment of time, lots of experimentation, and the use of one or more high blood pressure medications.

One of my recent goals is to help people better understand how to use AV Foundation. I started working on this last fall when I gave my “Learning AV Foundation” talk at 360|iDev. I’ll be giving an updated version of that talk this week at the Minnesota CocoaHeads meeting. I’ll also be giving a talk entitled “Composing and Editing Media with AV Foundation” at the CocoaConf stops in DC and San Jose in the coming weeks. As part of both talks I have sample apps that I’ll be releasing on GitHub that will illustrate how to make use of the framework.

I’ve got some additional things planned for later this year and will post more about those when I get a bit closer. I like to do at least one useful blog post a year so, who knows, maybe I’ll even find the time to write about the topic on my blog.

This past spring I developed an introductory training course called Learning iOS Programming for Infinite Skills. Given my background in training I had been considering developing a course like this for some time and when the opportunity came along I was eager to jump on it. Developing and producing the course was no small effort, but I’m glad I did it and am happy with the results. I hope it will be a useful resource for developers moving to the iOS platform.

The course is now officially available from the Infinite Skills website and you can also find it on O’Reilly’s Safari Books Online. Check it out and let me know what you think.

I mentioned that this is my first video training course, but it definitely won’t be my last. My company, TapHarmonic, has been hard at work on some new offerings that we’ll hopefully be ready to reveal later this year. Stay tuned!

One of the great new features of iOS 5 is the addition of the Core Image (CI) framework. This framework has been available for a few years on the Mac, but now its advanced image processing and filtering capabilities are available on the iOS platform as well. One particularly cool new feature available in Core Image, on both iOS 5 and Lion, is its ability to detect faces in an image. Thanks to some recent inspiration from Jon Steinmetz over at Pixel Research Labs I have put together this tutorial showing how to use this great new feature.

Getting Started

The face detection API is surprisingly simple to use. It really boils down to two classes: CIDetector and CIFaceFeature. CIDetector is responsible for performing the analysis of an image and returns a collection of CIFaceFeature objects describing the face(s) found in the image. You begin by creating a new instance of CIDetector using its detectorOfType:context:options class method.

CIDetector can currently only be configured to perform face detection so you’ll always pass the string constant CIDetectorTypeFace for the type argument. The context and options arguments are optional, but you will typically provide it an options dictionary describing the accuracy level to use. This can be configured by defining a dictionary with the key CIDetectorAccuracy and a value of either CIDetectorAccuracyLow or CIDetectorAccuracyHigh. The high accuracy algorithm can produce far more accurate results, but takes significantly longer to perform the analysis. Depending on what you need to accomplish you may find the low accuracy setting produces acceptable results.

Analyzing the Image

With a properly configured detector in hand you’re ready to analyze an image. You call the detector’s featuresInImage: method passing it an image to analyze. The Core Image framework doesn’t know anything about UIImage so you can’t directly pass it an image of this type, however, UIKit provides a category on CIImage making it easy to create an instance of CIImage from a UIImage.

The featuresInImage: method will return a collection of CIFaceFeature objects describing the features of the detected faces. Specifically, each instance defines a face rectangle, and points for the left eye, right eye, and mouth. It only defines the center point of each feature so you’d have to perform some additional calculations if you’d need to know the feature’s shape, angle, or relative location.

Visualizing the Results

The following images show examples of the face detection API in action. The images illustrate the differences between the low and high accuracy settings along with the approximate times it took to run the detection. The location of the detected features is not significantly different between the two images, but you’ll notice the high accuracy setting took more that 10x longer to compute on an iPhone 4. It will likely require a fair amount of testing of a representative set of images to determine the appropriate accuracy setting for your app.

CIDetectorAccuracyLow

~253ms on iPhone 4

CIDetectorAccuracyHigh

~ 2810ms on iPhone 4

I have put together a sample app containing images of several iconic faces. Flip through the images and run the analysis to see the face detection in action. You can run the sample on the simulator, but I’d recommend running it on your device so you can get a realistic sense for the performance. Enjoy!