Benchmarking Android Data Leak Detection Tools

Abstract

In 2017, Android hit a global mobile market share of 88% which makes it the most popular mobile platform. Application stores, such as the Google Play Store, are offering millions of mobile applications to consumers, which are installed and updated on a daily basis. However, the security of those applications is a major concern. A thorough security analysis before the publication of each application is time and resource consuming. Hence, platform providers cannot and do not manually vet every application handed in for publication. Consequently, many malicious and vulnerable applications find their way to the app stores and through there to the end users’ devices. Those applications exhibit serious security issues, such as leaking of sensitive information. During the previous years, researchers proposed a myriad of techniques and tools to detect such issues. There also exist large scale taxonomies classifying such tools into different categories. However, it is unclear how these tools perform compared to each other. Such a comparison is almost infeasible, since most tools are no longer available or cannot be set up any more. In this work, we review static analysis tools for detecting data leaks in Android applications. Out of 87 tools in the vulnerability detection domain, we are able to obtain 22 tools. We then identify 5 tools in the data leak detection domain and run them. We run them on a given data set with known data leak vulnerabilities and compare their performance. Furthermore, we run the tools on a larger set of real-world applications to assess the prevalence of data leak issues in open-source Android applications. We propose our own approach — DistillDroid — to compare security analysis tools by normalising their interfaces. This simplifies result reproduction and extension to other security vulnerability domains. In addition, the user experience and usability is highly improved.