For each conditional instruction, an execution trace only covered one branch, and the other branch indicates a new execution path not covered

Uncovered path predicates will be given to the constraint solver to see whether it can be satisfied by changing the execution environment

Path Selecting Heuristic

Bitblaze

TEMU

Record execution trace

Dynamic Analysis component of Bitblaze analysis platform

Dynamic Taint Analysis

Taint the environment related memory and the output of sensitive system APIs, which might be used by anti-unpack techniques e.g. PEB

Monitors the taint propagation at run-time and writes the tainted instructions into the trace

Symbolic Execution

In execution trace, each conditional jump depending on tainted memory or API output, will be generated two path predicates: one for the current path continuing with the true branch, and one for the current path continuing with the false branch

VINE

Static analysis component of the BitBlaze

Disassemble the trace and rransform the disassembly language into a platform-independent IR

Extract the constraints from the branch conditions by symbolic execution on the trace

Binary analysis platform

Unpacking tools

Approach

Uses the dependence of control flow of packed malware on execution environment to improve the resilience of dynamicunpacking tools

Steps

Starts with a fully correct and detailed program execution trace, if this satisfy malware unpacking requirement, then further analysis about how to change the execution environment is not needed

Symbolic execution component observes the trace recorded to search the computations which are affected by the environment, such as system time ,operation system properties and so on,

Builds a logical formula describing the relationship between the control flow of malware and the current execution environment. The logical formula is composed of many branch constraints.

Branch constraints are then systematically negated and solved with the constraint solver to see whether the new logical formula can be satisfied, indicating whether the corresponding new execution path are feasible

Approach then selects the next path that most likely to trigger unpacking behavior of malware from the set of feasible paths for further analyzing.

Collect branch conditions corresponding to execution environment, and predict the possible execution environment in which unpacking routine might be executed

Change the controlled environment to make malware unaware of the existence of the unpacking tool

What is?

Execution environment

PEB

Structured Exception Handling

Many malwares are packed to complicate malware analysis and evade malware detection

But hidden code or its equivalent must eventually be present in memory and get executed at run-time, unpacking tools is build around this nature

Dynamic unpacking tools are all based on a highly controlled environment to passively or actively monitor the written-then-executed behaviors of malware

Controlled environment will significantly increase the execution overhead that exposes the existence of unpacking tools. With the help of anti-unpacking techniques, malware may able to choose to remain dormant

Generally there must be a condition satisfied by the execution environment that lead the malware/hidden code to be executed

Anti unpacking techniques

Authors

Zhi Wang

Xin Liu

Chunfu Jia

Xinhai Liu

Kai Lu

Goals

Problem

Unpacking methods are all based on a highly controlled environment that is vulnerable to anti-unpacking techniques

Focus on anti unpacking methods based on API outputs and values of specific memory.

Contribution

Propose a new path-selecting algorithm to find a appropriate environment from many possible execution paths

Propose a directed hidden code extractor that is directed by the internal logic of the environment sensitive malware, and changes the environment to satisfy the requirements mimposed by malwares