How to Write a Cocoa Web Server

Editor's Note:Jürgen Schweizer is a new contributor to Mac DevCenter. Jürgen is president of Cultured Code, a small software company he founded in 2004. Its first product is the award-winning web design tool Xyle scope.

A couple of weeks ago a friend asked me to write a Mac-based HTTP server for an intranet web service he was building. I hadn't forgotten how Mac OS X was praised because of its Unix underpinnings, and the strong networking capabilities that come along with that. As a Cocoa developer, I was confident that I could find a nice Cocoa API that would do the hard work for me, so I accepted...

There are many powerful Cocoa APIs indeed, but none of which seemed to be exactly addressing the problem of writing a small homegrown web server. For example, there is the Distributed Objects (DO) framework that allows objects to send messages to each other through the network. Although extremely convenient, it is limited to communication between Objective-C objects, and cannot be used to build a more general server. In fact, it turned out that it was not possible to realize the HTTP server I sought without using some Unix and Core Foundation APIs as well.

However, there are some rather useful, although somewhat non-obvious, Cocoa classes that are most helpful for writing a web server. In this article I will show you what they are and how to use them. I will also show you when it is necessary to (or when some additional benefits can be gained) use Unix or Core Foundation APIs. This article assumes a basic knowledge of Cocoa, so I won't explain any Cocoa topics here, but will show everything you need to get started with networking. To have a concrete example at our hands, we will use the general classes we are building to create a server that accepts a URL and instead returns a PDF rendering of the web page at the given address. The full source code of the Xcode project is available at culturedcode.com/cocoa.

NSFileHandle and Network Communication

One of the problems in writing server software is the need to make sure that the server is always able to accept new requests no matter how busy it is fulfilling the old ones. Usually this requires the use of multiple threads. But fortunately, Cocoa provides us with a class that relieves us of creating a multi-threaded application: NSFileHandle. Yes, that's right. Despite the word "file" in the class' name, and the fact that it is not listed under Networking in Apple's documentation, it is just the right choice for handling all of our network communication needs. The explanation for this odd naming lies in Mac OS X's Unix foundation.

In Unix, pretty much everything is treated as a file. Physical devices, network connections, and yes, also ordinary files on your hard disk are all accessed using a common abstraction: file descriptors. A file descriptor is simply an integer that provides an index into a per-process table that references resources available to your process. Once we have the file descriptor fd of an already available resource, we can use it to create a new NSFileHandle object, like in the following code fragment:

NSFileHandle does not create network resources for us, but manages existing ones. For example, it will listen in the background for new connections or incoming data, notifying its delegate when appropriate. This capability of NSFileHandle is relieving us of spawning a new thread to do that work. Passing in YES for closeOnDealloc: makes sure that the resource managed by fileHandle is properly closed when we are finished with it.

But how do we get an appropriate network resource and corresponding file descriptor? The Unix abstraction for network resources is called BSD Sockets, and we will see shortly how to get our hands on one.

But first we need to choose on which port to make our service available to the outside world. Remember that a computer in a network is typically available at a specific IP address, which is commonly specified using four numbers (in the range from 0 to 255) separated by dots, e.g., 127.0.0.1. However, a single computer may provide many different services. To make this possible a single IP address is further divided into 65,536 different ports. All incoming messages specify a destination port. An application (or rather a process) can bind to specific ports, which means that incoming communications on those ports will be forwarded to the bound process.

For experimentation it is advisable to choose one of the unregistered ports (49152-65535). Ports can be registered with the Internet Assigned Numbers Authority (IANA).

The Easy Way to Sockets

To receive communications, we need to create an entry point in our process where data can flow in, and bind it to the IP address of our computer and the port number we have chosen. Such entry points are known as BSD sockets, or simply sockets, and are identified via file descriptors.

Again, Cocoa has a class that handles the necessary details for us. This class is known by the not so surprising name of NSSocketPort. This time, however, the gotcha is that its documentation is buried deeply within the Distributed Objects API, so much so that some developers even doubt that it can be used reliably for purposes other than that of serving the DO architecture. Rest assured that NSSocketPort will do a nice job for us. For all those still in doubt, the end of this article shows how to replace NSSocketPort with direct Unix calls and why one might want to do so.

PORT_NUMBER is the port number of our service (0...65535). It is important not to release socketPort before being finished with fileHandle.

Now that we have established a connection to the outside world, it would be nice if we could actually start accepting incoming connections. And we are only one method call away from reaching this goal:

[fileHandle acceptConnectionInBackgroundAndNotify];

This method call immediately returns, but fileHandle will keep listening to the socket until the first connection comes in. Then it issues an NSFileHandleConnectionAcceptedNotification, which carries further information about the incoming connection.

SimpleHTTPServer

To wrap up we will create a SimpleHTTPServer class and put everything we have learned so far into an initWithPortNumber:delegate: initializer method as follows:

The requests and connections instance variables will be used to keep track of open connections and pending requests, while delegate holds a pointer to an object responsible for doing the actual processing necessary to fulfill the request. In our example it will render a web page to PDF.

Once a SimpleHTTPServer object is created it will immediately begin listening to the specified port for incoming connections. As soon as a client begins to talk to our computer on the right port, NSFileHandle will issue an NSFileHandleConnectionAcceptedNotification which causes our newConnection: method to be called, since this is what we specified when we registered with NSNotificationCenter.

The implementation of the newConnection: method is as follows. Note that we are already using a soon to be declared SimpleHTTPConnection class. While SimpleHTTPServer has the responsibility to accept newly connecting clients, all communication from a specific client will be channeled through a corresponding SimpleHTTPConnection object:

The most notable thing here is the fact that through NSNotification's user info dictionary we are passed another instance of NSFileHandle. This new file handle represents the specific client who has connected to our service. We are using it to initialize a new SimpleHTTPConnection object. It is important not to forget to call acceptConnectionInBackgroundAndNotify on fileHandle again, since NSFileHandle will otherwise stop listening for incoming connections.

As a bonus, the code presented here makes sure it plays nicely with Cocoa bindings. It is possible to bind the connections instance variable to an NSArrayController, which in turn can easily be hooked up to a user interface widget, all from within Interface Builder. Bracketing our changes of the connections array with explicit calls to the observer notification methods willChange:valuesAtIndexes:forKey: and didChange:valuesAtIndexes:forKey: makes sure that the bindings layer will take notice.

Usually, such notifications are created automatically when instance variables are set. Here, however, we are not setting the connections instance variable itself, but we are mutating the referenced array. This will go unnoticed by the Cocoa bindings layer as long as we are not either using so called indexed accessor methods or are creating observer notifications manually. We have chosen to do the latter. Care has to be taken though, to not mix automatic and manual notification. Our newConnection: method will never participate in a binding. If it were, we would have to disable automatic observer notification first, for otherwise unexpected behavior and even crashes could result. More details on manual versus automatic observer notification can be found in Apple's Key-Value Observing Programming Guide.

SimpleHTTPConnection

In general, we have to assume that several clients will have open connections to our server at the same time. Each client might issue several new requests while our server is busy fulfilling old ones. We solve this problem by creating a new SimpleHTTPConnection object each time a new client connects to our service. Each SimpleHTTPConnection object will then take care of receiving incoming requests from the specific client connection it manages. In this way requests can always be easily associated with their connections.

Just like when initializing SimpleHTTPServer objects, we also register for appropriate notifications here. But this time we ask for NSFileHandleReadCompletionNotification notifications instead, and we also specify the file handle of the connecting client to make sure that we only receive notifications pertaining to the file handle we actually manage:

In the code above we have already prepared two instance variables message and isMessageComplete. These will be used shortly to make sure that we always receive complete HTTP messages.

Working with HTTP Requests

Through NSFileHandle we will only receive raw data. But since we want to talk to web browsers or other clients that use the HTTP protocol, we need to be able to both interpret and create HTTP messages.

Unfortunately, Cocoa has no API for working directly with the HTTP protocol. But it turns out that Core Foundation does. While Core Foundation uses the C language instead of Objective-C, its design is still object-oriented in nature. Indeed many Cocoa classes are wrapping functionality that is already implemented on the Core Foundation level.

Many Core Foundation objects can be used where Cocoa objects are expected and vice versa by simply casting the corresponding pointer. In Apple's documentation this is referred to as "toll-free bridging." Examples we will be using below are the NSURL and NSData classes. For example, pointers to NSURL objects can be used wherever a CFURLRef is expected. But toll-free bridging is not available for every Cocoa class. Developers should always consult the documentation first.

To interpret incoming data as HTTP messages, we will be using the following Core Foundation functions: CFHTTPMessageCreateEmpty, CFHTTPMessageAppendBytes, CFHTTPMessageIsHeaderComplete, and CFHTTPMessageCopyRequestURL. In general, there is no way to guarantee that a complete HTTP message will be received in one single piece. To deal with this, we will first create a new empty HTTP message:

message = CFHTTPMessageCreateEmpty(kCFAllocatorDefault, TRUE);

and then append received data to it as long as CFHTTPMessageIsHeaderComplete returns FALSE.

The CFHTTPMessage object message is created in the process' memory. We own Core Foundation objects that are created with functions containing "Create" or "Copy" in their names. Such objects need to be released using CFRelease.

HTTP messages come in two flavors: HTTP requests and HTTP responses. HTTP requests are sent to a server, which in turn replies with a HTTP response. By passing TRUE for the second argument of CFHTTPMessageCreateEmpty we specify that we are dealing with a request.

Once a HTTP request has been received completely, we can extract the request-URL using CFHTTPMessageCopyRequestURL(message). The details are as follows:

This time NSNotification's user info dictionary contains an NSData object that encapsulates the received bytes.

Note that delegate (of class SimpleHTTPServer) is assumed to respond to the methods newRequestWithURL:connection: and closeConnection:. The implementation of these methods is a routine task. SimpleHTTPServer collects all requests in an NSMutableArray. Every entry is a NSDictionary that records the URL, connection, and date of the request. Whenever a client aborts a connection, SimpleHTTPServer makes sure that every pending request pertaining to this connection is removed. The reader is referred to the source code for more details.

Let's step back for a moment and take look at the whole picture:

AppController

Our example Xcode project contains an AppController class that manages the complete application infrastructure. On application launch it creates a single SimpleHTTPServer object:

Note that AppController passes itself as the delegate. While the SimpleHTTPServer and SimpleHTTPConnection classes insulate AppController from networking details, the actual processing of requests is done by AppController itself. SimpleHTTPServer sends the following methods to its delegate to coordinate the processing of requests:

SimpleHTTPServer makes sure that only one request is sent to its delegate for processing at any time. If a request that has already been forwarded to AppController is aborted by the client, SimpleHTTPServer sends the stopProcessing method to notify its delegate accordingly. SimpleHTTPServer will not send further requests for processing until either the request has been aborted or one of the replyWith... methods has been called.

Sending HTTP Responses

SimpleHTTPServer handles the replyWith... methods by using the following more general method:

The method signature already reflects the fact that every HTTP response consists of a status line (containing a status code and a reason phrase), optional header fields, and the bulk of the message contained in the body. The most well-known status code probably is 404 (reason phrase "Not Found"), which signifies that the server was not able to find anything matching the request-URL. We use 200 for successfully completed requests. A full list of codes can be found in RFC2616's Status Code Section.

Core Foundation has a number of functions that make it easy to construct a HTTP response. We begin with creating a CFHTTPMessage object:

Then we add header fields to it. Header fields contain meta data about the bytes being sent. Every header field consists of a field name and a field value. For example, to add a header field that specifies the MIME type of the message body we use:

The CFSTR() function constructs a CFString object from a C string. CFString is toll-free bridged with NSString. The following function call adds AppController's data to the response:

CFHTTPMessageSetBody(msg, (CFDataRef)body);

Once we have finished constructing the HTTP response, we need to serialize it for transmission over the network:

CFDataRef msgData = CFHTTPMessageCopySerializedMessage(msg);

The raw data we have thus obtained can be sent to the client by using the writeData: method on the client's connection file handle. We have to exercise some care here, since it is possible that the network connection fails while transmitting our serialized HTTP response, in which case NSFileHandle raises an exception. It suffices to enclose the method call within a @try block:

Replacing NSSocketPort with Unix Calls

When testing code that uses BSD sockets there is one common problem. When the tested app crashes or is forced to quit for whatever reason, the socket still exists in the system kernel, and might stay there for a couple of minutes until a timeout kicks in. Rerunning the app before the socket has been freed results in an error. Using the following Unix call we can tell a socket to reuse the given address, thus effectively preventing this error:

Here fd is the file descriptor of the socket we are interested in. setsockopt() is a general interface for setting socket parameters. The problem here is that we need to call setsockopt() before fd is bound to an address/port combination. Thus we cannot use it after creating a NSSocketPort. Instead we have to completely replace NSSocketPort with direct Core Foundation and Unix calls as follows:

After this piece of code executes successfully, fd will contain the valid file descriptor of our server socket.

We have used three Core Foundation functions here, CFSocketCreate, CFSocketGetNative, and CFSocketSetAddress. But the constants we used and the sockaddr_in structure are taken from Unix. We won't go into the details here, but instead refer to an article by Mike Beam, who did an excellent job explaining BSD sockets.

Conclusion

We have seen how to use NSSocketPort, NSFileHandle, and Cocoa notifications to write a server application without having to create a multi-threaded application. We have seen how Core Foundation relieves us of directly working with the low level details of the HTTP protocol. Finally, we have written some code which helps to take away the frustration of having to wait for sockets to be freed by the system kernel. A complete working example, that also demonstrates how to use WebKit for rendering a web page to PDF, can be found at culturedcode.com/cocoa.