An Experiment in Developing Small Mobile Phone Applications
Comparing On-Phone to Off-Phone Development

TouchDevelop represents a radically new mobile application
development model, as TouchDevelop enables mobile application
development on a mobile device. I.e., with TouchDevelop, the task of
programming say a Windows Phone is shifted from the desktop computer
to the mobile phone itself. We describe a first experiment on
independent, non-expert subjects to compare programmer productivity
using TouchDevelop vs. using a more traditional approach to mobile
application development.

(2) Program used to build TouchDevelop application corpus

Disclaimer: The design of the TouchDevelop cloud web services is
still under heavy development. The APIs may change at any time and and
may thereby break.

Our tool also counts LOC as described below. Usage:

> java -jar TouchDevelopAllAppsReport.jar > report.txt

(3) How large are TouchDevelop applications?

To answer that question, we used our TouchDevelopAllAppsReport
tool to download from the TouchDevelop cloud all apps that have ever
been submitted. From the LOC counting we exclude all prior versions of
an app published by the app's original author under the same app name.
By doing that we only count the current version of each app. However
we do not exclude such apps if the prior and current app are by
different authors or have different names.

Size
in LOC of the 2,081 TouchDevelop apps published in the TouchDevelop
cloud as of 17 February 2012. Not shown are the two largest apps of
1,742 and 1,675 LOC. Each dot represents all apps in one bin of 5
LOC. E.g., the left-most dot represents the 674 apps from 0 to 4 LOC.

As of 17 February 2012, the largest TouchDevelop app is
Photo Dog with 1742
LOC.

(4) Counting LOC

To make counting reproducible and comparable between TouchDevelop and
Android, we first normalized all TouchDevelop and Android apps and
then counted their logical source statements (LSS) http://www.sei.cmu.edu/library/abstracts/reports/92tr020.cfm.
We did not count the content of configuration files such as the xml
files each Android app uses to define layout, styling, etc.

To count individual TouchDevelop apps of each subject, we first
publish them to the TouchDevelop cloud and then use our following
TouchDevelopService tool to download them and count their LOC. You can
download the TouchDevelopService tool at TouchDevelopService.jar.
TouchDevelopService takes as parameter a list of subjects' userid and
counts their apps' LOC. For example:

To count individual Android apps, we create a custom Eclipse code
formatter that imitates the layout rules we used for counting the
TouchDevelop LSS. The formatter configure file can be download at org.eclipse.jdt.core.prefs.
In order to apply the formatter configure file on Android_src, first
download Eclipse IDE for Java Developers, then run the below command
inside the Eclipse application folder.

- After source code is formatted, we develop a tool which uses
JavaNCSS (http://javancss.codehaus.org) to count the Java LOC in each
Android project. You can download the tools at AndroidCodeCounter.zip and run
the command below on the folder that contains the extracted
Formatted_Android_src.zip.

> java -jar AndroidCodeCounter.jar
Formatted_Android_src

(5) Lab experiment comparing on-phone to off-phone development

We defined a set of 11 simple programming tasks. For the
experiment, Microsoft Research loaned us 10 mobile phones, which
allowed us to have subjects develop TouchDevelop apps on a phone. We
randomly assigned 10 subjects to one phone each and the control-group
of the remaining 17 subjects to a lab Windows PC each that had
installed an Android IDE (Eclipse) and SDK. The Android IDE contains
an interactive wizard that generates a working hello-world Android
app. The experiment used TouchDevelop v2.4.0.0 beta and Android v1.6.

TouchDevelop subjects submitted the following solutions. Each link in
the following list links to all solutions submitted by one of our 10
TouchDevelop subjects: wivo,
fvfi, mzxo, dwns, ybqh, askf, kgbv, lavd, snlg, kzhv.

Acknowledgments

This material is based upon work supported by the National Science Foundation
under Grants No. 1017305 and 1117369. Any opinions, findings, and conclusions or recommendations
expressed in this material are those of the author(s) and do not necessarily reflect
the views of the National Science Foundation.