100 COMMUNICATIONS OF THE ACM | FEBRUARY 2017 | VOL. 60 | NO. 2
have been sent and received, what cryptographic materials
have been computed, which messages are expected next,
etc. The state machine for each individual ciphersuite is
specified in the standard, but the task of writing a composite state machine for multiple ciphersuites is left to each
implementation.

Figure 2 depicts a simplified TLS handshake for some
(fictional) ciphersuite, as seen from the viewpoint of the
client. On the left, the client first sends a Hello message
containing a list of supported ciphersuites to the server.
The server chooses a ciphersuite and responds with two
protocol messages, A and B, to establish a session key for
this ciphersuite. The client completes the handshake by
sending a Finished message to confirm knowledge of
the session key. At the end of the handshake, both the client and server can be sure that they have the same key and
that they agree on the ciphersuite. Now suppose we wish
to support a new ciphersuite, such that the client receives
a different pair of messages, C and D, between Hello and
Finished. To reuse our well-tested code for processing
Hello and Finished, it is tempting to extend the client
state machine to receive either A or C, followed by either
B or D. This naive composition implements both ciphersuites, but it also enables unintended sequences, such
as Hello; A; D; Finished. In TLS, clients and servers
authenticate the full message sequence at the end of the
protocol (in the Finished messages) and, since no honest server would send D after A, allowing extra sequences
at the client may seem harmless.

However, a client that accepts this message sequence is
actually running an unknown handshake protocol, with a
priori no security guarantees. In our example, the code for
processing D expects to run after C has been received. If C
contains the server’s signature, then accepting D without
C may allow a crucial authentication step to be bypassed.
Furthermore, the code for processing D may accidentally
use memory that should have been initialized while processing C. Such memory safety bugs can lead to dangerous
attacks such as HeartBleed,a which exposed the server’s
internal state and private keys to remote attackers.

1. 2. Testing for state machine flaws

In Section 2, we describe a methodology for systematicallytesting whether a TLS client or server correctly implementsthe protocol state machine. We find that many popularTLS implementations exhibit composition flaws like thosedescribed above, and consequently accept unexpected mes-sage sequences. While some flaws are benign, others leadto critical vulnerabilities that a network attacker can exploitto bypass TLS security. In Section 3, we show how a networkattacker can impersonate a TLS server to a buggy client,either by simply skipping messages (SKIP) or by factoringthe server’s export-grade RSA key (FREAK). These attackswere responsibly disclosed and have led to security updatesin major web browsers, servers, and TLS libraries.

1. 3. Formally verifying state machine code

We have seen that the security of a TLS implementation
depends crucially on its correct implementation of the protocol
state machine. Testing can help find some bugs, but once these
have been fixed, how can we be sure that the code does not have
other hidden flaws? We advocate the use of formal verification
to prove the absence of any state machine flaws. In Section 4,
we present a new state machine implementation for OpenSSL
that supports all commonly enabled ciphersuites, versions, and
extensions. Using Frama-C, 11 we verify that our code conforms
with a logical specification of the TLS protocol state machine.

1. 4. Online materials

We refer to https://mitls.org/pages/attacks/SMACK for additional details, including security testing tools, a summary
of vulnerability disclosures and security updates, our state
machine code for OpenSSL, and related verification work on
TLS.

2. TESTING THE TLS STATE MACHINE

The TLS standard14 does not define a state machine. Instead,
it specifies a collection of message sequences, one for each
handshake protocol mode. Other specifications add new
ciphersuites, authentication methods, or protocol extensions; they typically define their own message sequences,
reusing the message formats and mechanisms of TLS, and it
is left to the implementation to design a state machine that
can account for all these sequences.

2. 1. A TLS state machine

We propose a reference state machine for TLS by adopting
and extending the one used in the mi TLS verified implementation, 6 based on a careful reading of the standard. Figure 3
depicts a simplified version of this state machine, which can
be read from the viewpoint of the client or the server. Each
state refers to the last message sent or received; messages
prefixed by Client are sent by the client; those prefixed by
Server are sent by the server. Transitions, shown as black
arrows, indicate the order in which these messages are
expected. When two transitions are possible, each is labeled
by the condition under which it is allowed. (Dotted arrows
are flawed transitions; they will be explained in Section 3.)
The state machine depicted here covers the common usages
of TLS on the web, a small but important subset of the full
protocol. The figure only shows message sequences; it does
not detail message contents, local states, or cryptographic