Monday, May 25, 2009

Most iPhone applications communicate with a server to perform their functions. This post is a step-by-step guide for snooping on the communications between the iPhone application and its server. The instructions can come in handy for debugging your own application, or if you're curious how other applications communicate with their servers.

Overview
My method uses Wireshark's WiFi RadioTap promiscuous mode to capture all the radio traffic, and find the iPhone's traffic. This article starts by laying out the necessary "ingredients", and guides you through setting up your iPhone and your Mac. Snooping the traffic is demonstrated by snooping on Apple's Stocks application. The post wraps up by describing my motivation for snooping on iPhone apps.

Ingredients
This post is tailored to my home environment, which is described below. Most differences between that environment and yours can be compensated by a bit of creativity. Here's what you need:

iPhone or iPod Touch. As long as it can run the application and connect to WiFi, it works.

Open WiFi network. Most schools and work places have open guest networks, which work for this purpose. At home, I disable the security on my router/AP for the duration of my snooping, and re-enable it later on.

Mac computer with OSX Leopard. It may work on Tiger, I haven't tried. It may work on hackintoshes, but I haven't tried that either. The software I'm using also has Windows/Linux ports, which I haven't tried.

iPhone / iPod Touch Setup
The iPhone can communicate using the cellular network, in addition to the WiFi. We want to make sure that doesn't happen. The fastest way I know is to go to Settings and enable Airplane Mode, and then select and enable WiFi and re-connect to the access point. If you are using an iPod Touch, you don't have to worry about this: it can only communicate via WiFi.

Snooping on applications is a lot easier if you know your iPhone / iPod's IP address. To find the IP launch Settings, and select WiFi, click on the blue arrow next to your access point's name, and read the IP from under the DHCP tab. This blog post has a thorough guide for this step, with pictures.

Computer Setup
Go to Wireshark's download page and download the stable version .dmg for your computer. The stable version at the time of this writing has all the necessary features for snooping, so you don't need the development version unless you feel adventurous. Yes, I knew you'd ask!

The Wireshark installation is not straightforward yet (this writing uses version 1.12), so I will go through the steps. Start off with the easy part, and drag the Wireshark icon to the Applications folder. The following commands (which you can copy-paste in Terminal) implement the instructions in the Readme.rtf included in the .dmg download.

Application Traffic Snooping
Before starting Wireshark, make sure your Mac is using WiFi. I have both LAN and WiFi connections, and I pull out my LAN cable before starting up Wireshark.

Start Wireshark, ignore the dialog boxes (there should be one informing you about a potentially long startup time, and one about missing stuff while loading MIBs). Open the Capture menu, and select Intefaces. Identify your WiFi interface - it's usually en1 (that's always the case on a Macbook / Macbook Pro). Click the Options button, change the Link-layer header type to IEEE 802.11 plus radiotap WLAN header, and enable Promiscuous mode.

Your capture should be as short as possible, to make analysis easy. For this reason, get ready to launch your iPhone application, and launch it as soon as Wireshark starts capturing traffic. Click the Start button at the bottom-right of Wireshark's dialog to start playing.

To stop the capture, select Stop from the Capture menu. For a shortcut, you can use the 4th toolbar icon from the left. If everything went well, all the iPhone traffic is available for your analyzing pleasure.

The example below shows an easy method for looking at an application's traffic.

Apple Stocks Traffic
This section describes how you can observe the traffic of Apple's Stocks application that comes pre-installed on iPhone OS. You can safely skip it if you feel like exploring Wireshark on your own.

First, use the instructions above for capturing the iPhone's Internet traffic for a few seconds, right when Stocks is launched. In the packets table, click on the Source header to sort packets by source. Find the packets originating from your iPhone / iPod Touch. Go through until you find something interesting.

For the Stocks application, the first interesting packet is a DNS resolution request for iphone-wu.apple.com which is the server feeding Stocks its information. The packets right under that are TCP packets, and you can right-click on any of them and Follow TCP stream. You will see a HTTP request / response between the Stocks application and Apple's servers. The imei parameter there caused some uproar (and blog traffic) a couple of years ago, so traffic snooping can definitely pay off.

When you close the TCP stream window, your packets window will only show the packets related to the request / response pair that you just saw. If you look in the Filter field under the toolbar, you can get a glimpse of Wireshark's filter syntax. The filter can be edited. For example, if you remove the predicate consisting of tcp.port eq and some big number, you will have all the HTTP packets between exchanged between the iPhone / iPod Touch and Apple's server.

By now, you should have a good glimpse into Stocks' communication protocol. Of course, the method described here applies to any other application, as long as it doesn't use encryption (e.g. SSL / TLS).

Motivation
I use this method to see where iPhone applications get their data from, and how they communicate with their servers. For example, the Stocks application claims it uses Yahoo data, and I wanted to see if it has a private XML feed, or if it implemented its own JSON parsing.

I also used this method to analyze the protocol of an online game that I like, so that I can write a script for automating the boring tasks.

Thank you for reading this post! I'm looking forward to your feedback. I would especially appreciate comments on simplifying the setup process. Happy snooping!

Wednesday, May 13, 2009

I have recently open-sourced the ZergSupport code updates used in StockPlay versions 0.3 and 0.4. The high-visibility high-impact change is support for JSON parsing. This post shows what you can do with JSON parsing.

To sweeten the deal even more, ZergSupport's JSON parser was extended so you can conveniently embed literals in Objective C strings. First, strings can be delimited by ' (single quotes) asides from the standard delimiter " (double quotes). Second, the extended parser understands sets, which look like arrays, but are delimited by angled brackets ( < > ) instead of square brackets ( [ ] ). Without further ado, here's how to use JSON literal support.

A small wrinkle to be aware of is that JSON parsing is slower than building the collections directly in Objective C, so JSON literals should be used in features that are not performance-sensistive, like tests and configuration files.

Web Services
The API for working with JSON Web services is very similar to the API for XML services, which is showcased in my first post on ZergSupport. When used from the Web service API, the JSON parser ignores everything up to the first { character, so it is able to parse JSONP output.

The code below is a complete implementation for stock ticker symbol search, using Yahoo Finance. The code uses Object Query (covered below) to indicate which parts of the JSON response should be transformed into ModelSupport models.

Object Query
Object Query implements a domain-specific language (DSL) for retrieving objects from a structure of deeply nested Cocoa collections. Queries are specified as strings, and are performed against the root object in a structure of nested collections. The result of a query is an array of zero or more objects matching the query.

The queries are property names, joined by a separator character (usually /). The first character in the query is the separator character. For example /results/1 matches the value that object['results'][1] would return in JavaScript. The special property names * and ? are inspired from glob expressions: * means the next property may be found in the current object or in some descendant object (some levels deeper in the object graph), whereas ? means the next property may be found in the current object or in the object's direct children (one level deeper in the object graph). For example /?/1 will return the same result as /results/1, if the initial object is {'results':['Jane','John']} (the result will be a NSArray containing the NSString @"John").

ObjectQuery can be used directly, outside of JsonHttpRequest, as shown in the following code sample.

The decision to introduce a DSL is motivated by the need to extract ModelSupport models from JSON Web service responses. XML objects have a tag/content separation, and tags are usually good indicators for model extraction purposes. For JSON objects, the closest equivalent to a tag name would be the property name whose value is the object hash describing the model. This is fragile, and does not work if the response contains an array of models. ObjectQuery is a bit more general than name tags, but does not degenerate into XPath's complexity. The implementation is 182 lines of Objective C, including comments and whitespace.

Conclusion
JSON support does not stop at the parser. The toolkit fully embraces JSON, which is now available for initializing ModelSupport models. The toolkit's new Object Query bridges the gap between XML-based and JSON-based Web Services, and preserves API consistency in WebSupport.

Sunday, May 10, 2009

This post is a short description of the community effort I'm trying to start around the iPhone application security model. It describes the effort, my motives for starting it, and the method I have chosen. The effort is hosted on George Hotz' theiphonewiki.com, with George's permission.

Effort

I have created an Application Copy Protection section on The iPhone Wiki. I hope that the wiki will become a place for developers to pool their knowledge on iPhone application security. In turn, this will make iPhone development less expensive and more enjoyable. Ideally, we would develop a code obfuscation method, as well as a server-side integrity check method, which are non-trivial to reverse. Once there is a barrier against automated programs and beginner crackers, piracy will hopefully go down to a more acceptable rate.

MotivationI'm dissatisfied with the asymmetry in the iPhone security landscape. On one hand, application pirates have a good infrastructure, ranging from tutorials to the Crackulous application for automated piracy, and to the Appulous infrastructure for distributing pirated applications. On the other hand, developers have to fight many unknowns, like the unspecified signature system, because Apple designed the system on the assumption that developers will not have to worry about copy protection themselves. Application security information is spread across Apple's documentation and various blogs and forums, which makes it hard for developers to learn and implement application security.

I'm also unhappy with RIPdev's approach of charging setup fees and royalties, because the application developers are already paying Apple an up-front development fee, as well as distribution fees.

Last but not least, I'm obviously a bit upset that my application got pirated the next day after it launched in the iTunes store :)

MethodI am documenting my thought process and method for establishing the effort for historical reasons. They will hopefully be useful to other people who want to start similar initiatives.

I wanted to do a wiki on iPhone application security, but I didn't take the time to think the logistics until recently. I was initially thinking of opening a Google Site, and adding as a collaborator any person that would e-mail me an useful piece of information. Then I realized that Google don't look as open to contributions as Wikis do. At the same time, I also started thinking about getting visibility for the site. After a bit of thinking, I realized I'm better off hosting the effort on The iPhone Wiki, because it's already a well-known site, its topic is security on the iPhone, and it contains information that can be useful to developers researching application security.

After I decided on The iPhone Wiki, I did some googling to find out that it was started by George Hotz, and I read the wiki's Constitution to see if my effort belonged there. I was still unsure if my effort fits in, so I decided to ask for George's permission. After some more googling, I eventually tracked him down, and he gave his consent quickly.

Having gotten George's consent, I spent a bit of time thinking of the best way to blend the topics I wanted to add with the existing content on the Wiki. I chose to create a separate section named Application Copy Protection on the front page, and created a skeleton under it. This optimizes for visibility, and makes it easy for me to optimize my thoughts, but may not be the best solution for the overall site. Fortunately, it's a Wiki, so I don't have to worry too much. If I made a mistake, someone else will jump in and fix it.

My next steps are:

contribute enough content to make the wiki worth reading for iPhone application developers

create a skeleton for what I think the rest of the content should be, so other people can easily jump in and contribute their knowledge

pitch the effort to high-traffic iPhone-related blogs, to make developers aware of the Wiki; the fact that the pages are hosted on The iPhone Wiki should help

ConclusionThis is my first grown-up attempt at starting a community effort. I would appreciate any suggestions or generic feedback. I hope you found the post at least amusing, if not useful.

Short Bio

Victor Costan got infected with a passion for coding at the age of 10, and proceeded to earn a M.Eng in Computer science from MIT. Victor also holds a B.S. in Management that he pursued as he fell prey to the belief that great software requires legions of developers manipulating bloated code bases in repetitive ways. Thanks to Paul Graham's essays, Ruby, and Rails, he re-discovered the joy of coding, and came back from the dark ways of management.

Nowadays, Victor is still crazy about building software that he can impress his friends with, but he also dreams of contributing to making coding even more enjoyable. Victor likes developing mobile and Web 2.0 applications, and occasionally enjoys building some distributed systems behind the apps. When he is not coding, you will most likely find him reading up on some new programming language or software development technique.

At times, Victor likes to pretend he's mentally sane, and can be seen catching up with friends, watching a movie, or exploring Boston.