iOS Tutorial: Advanced Networking with MKNetworkKit

Couple of weeks ago, I wrote a clean, fast networking toolkit for iOS and Mac written for the LLVM Compiler 3.0 with ARC.
Reception was very good that it was the “most-watched” repository on Github last week. Early adopters have sent me innumerable emails on how fast their network operations are, and how responsive their app is after integrating MKNetworkKit.

MKNetworkKit is faster (and it makes your app feel smoother and faster with seamless transparent caching) in reality. There are two things that makes MKNetworkKit faster.
First, it’s ARC based and the awesome LLVM 3.0 compiler brings in a ton of performance benefits. Using the @autoreleasepool block instead of NSAutoReleasePool in MKNetworkOperation alone should improve the performance by a huge factor (Apple claims it to be 6 times faster). If you still haven’t migrated to ARC because you want to support those iOS 3.x users, read this post by Matt Gemmell first.

Second important performance benefit is seamless and transparent caching. By transparent caching, I mean, caching that involves zero overhead from the developer (That’s you).

Let me, in this post, brief you about how this seamless caching might help you. First, an example. You are refreshing a twitter stream. A GET request for that looks similar to

GET http://twitter.com/mugunthkumar/statuses

Caching your responses, the wrong way

Some developers cache responses using Core Data. STOP IT

Using Core Data for caching is like using the military to kill bedbugs in your bedroom. DON’T DO IT.

The problem when you implement Core Data is, you should program your view controllers to work with two kinds of data structures. One, JSON/XML from server and the other, Core Data NSManagedObjects. Some “clever” developers “take it to the next level” by just programming their view controllers to work with only Core Data. The design goes like this. The network layer talks to the server and updates Core Data Store. Once the store is updated, send a NSNotification to “refresh” view controllers.
This may sound clever to many. But it’s completely WRONG. I REPEAT. COMPLETELY WRONG. Core Data (or even structured SQL storage) is not meant for that. Core Data is a structured storage for persisting/serializing Object graphs in your application. Don’t use it for storing objects that have a low life time. Thumbnail images, list of tweets etc., just don’t belong there. Secondly, this would require you to read and write to flash memory on the iDevice which have a limited read/write cycle. Finally, reading and writing to disk (in this case, Core Data) is a very ugly way to transfer data from one class to another (in this case, from your Network Layer to View Controllers)

Say hello to MKNetworkKit

MKNetworkKit calls the SAME completion handler with cached data if you are making the call for the second time. When the network connectivity is proper, MKNetworkKit calls your completion handler twice. First with the cached data and again after fetching the latest data from server. As such, you can design your view controllers to work with just ONE kind of data, data from server. If you have an app that doesn’t cache and doesn’t work offline, just replace your networking library with MKNetworkKit and you get caching for free. Not even a single line of code change is required on your view controllers. Caching is transparent. Your view controllers needs to work with the same data structure, no matter whether the data is from cache or server.

Behind the scenes

MKNetworkKit caching is super light-weight and it caches your responses in a NSMutableDictionary. Doesn’t that lead to high memory usage? Yes. of-course, but MKNetworkKit is clever. It observes the UIApplicationDidReceiveMemoryWarningNotification and flushes the dictionary to disk. The next time you make a request to the cached URL, the cached data is brought back to memory. So in-effect, it has a in-memory cache and disk cache and uses a least recently used algorithm to flush items to disk. The MKNetworkEngine class has methods that can be over-ridden to control the cache cost.

Just override the method

-(int) cacheMemoryCost;

-(int) cacheMemoryCost;

and return a higher/lower value based on your application’s requirements.

MKNetworkKit caching works like a intermediate proxy server by intercepting the response headers. So, when a second GET request to the same URL is made, MKNetworkKit behaves the following way.

1) For GET requests that sent a ETag in the header previously (for example, requests to Amazon S3 servers), MKNetworkKit sends a second request as a HEAD request (even if you specify as GET) with IF-NONE-MATCHheader. Most servers that send ETag (like Amazon S3) responds to IF-NONE-MATCH by sending the real data only when it has been changed. Otherwise, they return a 304 Not Modified which MKNetworkKit safely ignores. So data transfer is really not huge. Even this request is honored only after 1 day. So, repeated requests to the same thumbnail image doesn’t even hit the server.

2) For any GET request that contained a Cache-Control:max-age=in the header, MKNetworkKit honors that and makes the second request only after the expiry date. For dynamically generated requests, you should set Cache-Control on server. This not only helps MKNetworkKit, but most proxy servers along the way.

3) For requests that contained a Last-Modified field in the header, MKNetworkKit sends a “IF-MODIFIED-SINCE” HEAD request. This again works like the IF-NONE-MATCH. The server, either sends the complete data or a 304 Not Modified.

4) For servers that implement Cache-Control: no-cache, MKNetworkKit goes one step ahead and makes a second request only after a minute (60 sec). So repeated refreshes to often refreshed page will not exactly hit the server. For example, refreshing foursquare check-ins. It’s rare for these information to change within the next 60 sec.

5) For performance reasons, when the response type is an image, and the response headers doesn’t have any cache control information, MKNetworkKit assumes an expiry date of 7 days.

The last two are not an RFC standard. But these optimizations make network operations fast on a mobile devices even if the server isn’t implemented in an optimized way. All POST, PUT, HEAD and DELETE requests are ignored and not cached.

And, of course, you can customize all these behaviors by editing the values in MKNetworkKit.h

Freezing Operations

Another very important feature of MKNetworkKit is operation freezing. Imagine that, your customer take an awesome photo with your own “Instagram-killer” app, but since he is at a remote exotic place, 3G connectivity is poor and uploading the photo fails. Without MKNetworkKit, you should remember the photo related meta data, store the photo’s NSData in a file and upload the operation the next time the app is launched. Painful!
With MKNetworkKit, you mark the photo upload operation is “freezable”. This means, in the event of network failure, your operations are automatically serialized (frozen beneath snow) and restored the next time the app launches, all for free.

When your server uses HTTP Basic auth or Digest auth or even Windows NTLM authentication, all you need to do is to set your username and password and MKNetworkKit auto-magically authenticates your request!. For NTLM authentication, just ensure your username is “domain\username”. There is no separate method setDomain: or something.

You can also set basicAuth:YES to send the credentials even before receiving the authentication challenge. However, this works only if your server uses HTTP Basic Authentication.

Custom Authentication

If your server uses HTML Form authentication, you can override the authHandler block and provide custom auth mechanisms.

MKNetworkOperation *op =[myEngine operationWithPath:@"/letmein"];
op.authHandler =^(NSURLAuthenticationChallenge*cred){// show a web view controller or do whatever you want and finally, when you have a credentials ready, just call [challenge.sender useCredential:credential forAuthenticationChallenge:challenge];
};
[myEngine enqueOperation:op];

Overriding MKNetworkEngine

Tweaking the URL building methods with MKNetworkEngine is easy. Let me start by giving you an example. Imagine that you have a server which authenticates a logged in user and after authentication, it expects a Authorization header to be set with an access token.
You need to have a factory method that creates URLs customized to your web service. This could include adding a authorization header in our case. You can do this by overriding the method,

-(void) enqueueOperation:(MKNetworkOperation*) operation;

-(void) enqueueOperation:(MKNetworkOperation*) operation;

For example, in the above case, you can check your engine subclass for an access token and add it to the operation’s header in this method (if you have one).

Overriding MKNetworkOperation

Sometimes, you might do custom error handling based on the response from your server. For example, your server might send a valid HTTP response (status code 200) which could be an internal error, like “Invalid User”. Business logic errors are better handled with your own application level error codes. You overriding MKNetworkOperation to customize the success/failure reporting. I would highly recommend doing this along with designing your server to send error codes for error conditions.

The following methods are to be overridden for customizing error handling.

For example, in the previous case, you can override operationSucceeded, inspect the response and if the response was indeed a business logic error (“Invalid User”), you can call the [super operationFailedWithError:[Your custom error class for “Invalid User”]];
Otherwise, call [super operationSucceeded].

You can also override operationFailedWithError to introspect the actual cause of the error, if your server sends any error related information in the dictionary.

The reason for overriding at this level is to minimize error handling at view controller layer. So your presentation layer, UIViewController subclasses will be cleaner to read.

If you have subclassed the MKNetworkOperation and want the factory method in your engine subclass, operationWithURLString:params:httpMethod to prepare an operation of your subclass. To register your operation subclass with the engine, you can call the registerOperationSubclass: method.

Image Cache with MKNetworkEngine

MKNetworkEngine has a handy method to load images from a server. It’s built-in cache mechanism ensures, your images are cached without using any third-party image cache libraries. That means your code base is even leaner.
You can use the following method of MKNetworkEngine to load images.

Source Code

MKNetworkKit might look simple, but it’s incredibly powerful. Use it in your apps and tell me how it scores. My next blog post would be on how to write a better RESTful server that serves a mobile device. Watch this space, subscribe to my feed or follow me on twitter.

Interested in learning in-depth features of Objective-C and iOS? You should get my book.