There is a need for appropriate analysis approaches that allow
researchers to understand what malware, or generic unknown programs,
do on the target system. Hardware-based malware-analysis sandboxes
have been recently proposed to replace emulator-based sandboxes,
thanks to their transparency and resilience to emulator-detection
attacks. A core part of any sandbox is its capability of "tracing" a
(malicious) running program, such that the actions (e.g.,
instructions, operating system calls) that it performs on the system
can be observed. In state-of-the-art emulator-based sandboxes tracing
relies on so-called virtual machine introspection (VMI) techniques,
which consist in tracing the instructions from outside the virtual CPU
for reconstructing high-level events such as system calls. In
hardware-based sandboxes tracing is still an open problem, as it is
highly dependent from the debugging capability of the CPU.
Interestingly, we observe that the vast majority of mobile devices
(which are among the targets of malware authors) are based on the ARM
architecture, which natively supports machine-level debugging from
hardware interfaces.
In this work we assess the feasibility of implementing a system call
tracer for the Android/Linux operating system running on ARM-based
computers. We propose OpenST, an open-source tool that leverages
the JTAG interface to perform the equivalent of VMI yet in hardware.
More precisely, our tool uses hardware breakpoints to track the
occurrence of software interrupts and inspect the CPU registers in
order to reconstruct system calls. OpenST also inspects the
running process' memory to reconstruct the value of each argument
passed to the system function, performing pointer de-referencing and
data unmarshalling as needed. OpenST is portable across Linux
versions because it derives the system call prototypes from the kernel
binary image, from which it generates argument-unmarshalling
procedures automatically.
We implemented OpenST and evaluated its correctness against a
testing Linux application that invokes known system calls. Moreover,
we performed micro- and macro-benchmarks on 3 real-world applications.
Our micro-benchmarks show that the need for pausing and resuming the
CPU to inspect the memory for reconstructing the arguments values
imposes a substantial overhead, around 180ms, where a system call
takes 500--2000ns on average. In comparison the state-of-the-art
emulator based sandbox imposes an overhead of a fraction of the
millisecond. Our macro-benchmarks show that this overhead has an
impact of 70x on average on the overall execution time. In practice,
our tests with Android applications showed that this slowdown makes
the user interface unusable. We measured that the overhead depends
from the speed of the JTAG adapter, so, in principle, it could be
reduced by using faster hardware. In conclusion, I believe that our
approach is promising yet unfeasible with current low-cost hardware,
which is a requirement for large-scale malware analysis.