Abstract:
We propose a novel approach to infer protocol state machines in the
realistic high-latency network setting, and apply it to the analysis of
botnet Command and Control (C&C) protocols. Our proposed techniques
enable an order of magnitude reduction in the number of queries and time
needed to learn a botnet C&C protocol compared to classic algorithms (from
days to hours for inferring the MegaD C&C protocol). We also show that
the computed protocol state machines enable formal analysis for botnet
defense, including finding the weakest links in a protocol, uncovering
protocol design flaws, inferring the existence of unobservable
communication back-channels among botnet servers, and finding deviations
of protocol implementations which can be used for fingerprinting. We
validate our technique by inferring the protocol state-machine from
Postfix's SMTP implementation and comparing the inferred state-machine
to the SMTP standard. Further, our experimental results offer new
insights into MegaD's C&C, showing our technique can be used as a
powerful tool for defense against botnets.