Chapter 6. Variants

Variant types are one of the most useful features of OCaml and also
one of the most unusual. They let you represent data that may take on
multiple different forms, where each form is marked by an explicit tag. As
we'll see, when combined with pattern matching, variants give you a powerful
way of representing complex data and of organizing the case-analysis on that
information.

Each row essentially represents a case of the variant. Each case has an associated tag and
may optionally have a sequence of fields, where each field has a specified type.

Let's consider a concrete example of how variants can be useful.
Almost all terminals support a set of eight basic colors, and we can
represent those colors using a variant. Each color is declared as a simple
tag, with pipes used to separate the different cases. Note that variant tags
must be capitalized:

The following function uses pattern matching to convert a basic_color to a corresponding integer. The
exhaustiveness checking on pattern matches means that the compiler will warn
us if we miss a color:

In this example, the cases of the variant are simple tags with no
associated data. This is substantively the same as the enumerations found in
languages like C and Java. But as we'll see, variants can do considerably
more than represent a simple enumeration. As it happens, an enumeration
isn't enough to effectively describe the full set of colors that a modern
terminal can display. Many terminals, including the venerable xterm, support 256 different colors, broken up
into the following groups:

The eight basic colors, in regular and bold versions

A 6 × 6 × 6 RGB color cube

A 24-level grayscale ramp

We'll also represent this more complicated color space as a variant,
but this time, the different tags will have arguments that describe the data
available in each case. Note that variants can have multiple arguments,
which are separated by *s:

Once again, we'll use pattern matching to convert a color to a
corresponding integer. But in this case, the pattern matching does more than
separate out the different cases; it also allows us to extract the data
associated with each tag:

We've essentially broken out the Basic case into two cases, Basic and Bold, and Basic has changed from having two arguments to
one. color_to_int as we wrote it still
expects the old structure of the variant, and if we try to compile that
same code again, the compiler will notice the discrepancy:

Here, the compiler is complaining that the Basic tag is used with the wrong number of
arguments. If we fix that, however, the compiler flag will flag a second
problem, which is that we haven't handled the new Bold tag:

As we've seen, the type errors identified the things that needed to
be fixed to complete the refactoring of the code. This is fantastically
useful, but for it to work well and reliably, you need to write your code
in a way that maximizes the compiler's chances of helping you find the
bugs. To this end, a useful rule of thumb is to avoid catch-all cases in
pattern matches.

Here's an example that illustrates how catch-all cases interact with
exhaustion checks. Imagine we wanted a version of color_to_int that works on older terminals by
rendering the first 16 colors (the eight basic_colors in regular and bold) in the normal
way, but renders everything else as white. We might have written the
function as follows:

But because the catch-all case encompasses all possibilities, the
type system will no longer warn us that we have missed the new Bold case when we change the type to include it.
We can get this check back by avoiding the catch-all case, and instead
being explicit about the tags that are ignored.

Combining Records and Variants

The term algebraic data types is often used to
describe a collection of types that includes variants, records, and
tuples. Algebraic data types act as a peculiarly useful and powerful
language for describing data. At the heart of their utility is the fact
that they combine two different kinds of types: product
types, like tuples and records, which combine multiple
different types together and are mathematically similar to Cartesian
products; and sum types, like variants, which let you
combine multiple different possibilities into one type, and are
mathematically similar to disjoint unions.

Algebraic data types gain much of their power from the ability to
construct layered combinations of sums and products. Let's see what we can
achieve with this by revisiting the logging server types that were
described in Chapter 5, Records. We'll start by reminding
ourselves of the definition of Log_entry.t:

This record type combines multiple pieces of data into one value. In particular, a single
Log_entry.t has a session_idand a timeand an important flag
and a message. More generally, you
can think of record types as conjunctions. Variants, on the other hand, are disjunctions,
letting you represent multiple possibilities, as in the following example:

A client_message is a Logonor a Heartbeator a Log_entry. If we want to write code that
processes messages generically, rather than code specialized to a fixed
message type, we need something like client_message to act as one overarching type
for the different possible messages. We can then match on the client_message to determine the type of the
particular message being dealt with.

You can increase the precision of your types by using variants to
represent differences between types, and records to represent shared
structure. Consider the following function that takes a list of client_messages and returns all messages
generated by a given user. The code in question is implemented by folding
over the list of messages, where the accumulator is a pair of:

The set of session identifiers for the user that have been seen
thus far

There's one awkward part of the preceding code, which is the logic
that determines the session ID. The code is somewhat repetitive,
contemplating each of the possible message types (including the Logon case, which isn't actually possible at
that point in the code) and extracting the session ID in each case. This
per-message-type handling seems unnecessary, since the session ID works
the same way for all of the message types.

We can improve the code by refactoring our types to explicitly
reflect the information that's shared between the different messages. The
first step is to cut down the definitions of each per-message record to
contain just the information unique to that record:

As you can see, the code for extracting the session ID has been
replaced with the simple expression common.Common.session_id.

In addition, this design allows us to essentially downcast to the
specific message type once we know what it is and then dispatch code to
handle just that message type. In particular, while we use the type
Common.t * details to represent an
arbitrary message, we can use Common.t *
Logon.t to represent a logon message. Thus, if we had functions
for handling individual message types, we could write a dispatch function
as follows:

And it's explicit at the type level that handle_log_entry sees only Log_entry messages, handle_logon sees only Logon messages, etc.

Variants and Recursive Data Structures

Another common application of variants is to represent tree-like
recursive data structures. We'll show how this can be done by walking
through the design of a simple Boolean expression language. Such a
language can be useful anywhere you need to specify filters, which are
used in everything from packet analyzers to mail clients.

An expression in this language will be defined by the variant
expr, with one tag for each kind of
expression we want to support:

Note that the definition of the type expr is recursive, meaning that a expr may contain other exprs. Also, expr is parameterized by a polymorphic type
'a which is used for specifying the
type of the value that goes under the Base tag.

The purpose of each tag is pretty straightforward. And, Or, and
Not are the basic operators for
building up Boolean expressions, and Const lets you enter the constants true and false.

The Base tag is what allows you
to tie the expr to your application, by
letting you specify an element of some base predicate type, whose truth or
falsehood is determined by your application. If you were writing a filter
language for an email processor, your base predicates might specify the
tests you would run against an email, as in the following example:

The structure of the code is pretty straightforward—we're just
pattern matching over the structure of the data, doing the appropriate
calculation based on which tag we see. To use this evaluator on a concrete
example, we just need to write the base_eval function, which is capable of
evaluating a base predicate.

Another useful operation on expressions is simplification. The
following is a set of simplifying construction functions that mirror the
tags of an expr:

It fails to remove the double negation, and it's easy to see why.
The not_ function has a catch-all case,
so it ignores everything but the one case it explicitly considers, that of
the negation of a constant. Catch-all cases are generally a bad idea, and
if we make the code more explicit, we see that the missing of the double
negation is more obvious:

The example of a Boolean expression language is more than a toy.
There's a module very much in this spirit in Core called Blang (short for "Boolean language"), and it
gets a lot of practical use in a variety of applications. The
simplification algorithm in particular is useful when you want to use it
to specialize the evaluation of expressions for which the evaluation of
some of the base predicates is already known.

More generally, using variants to build recursive data structures is
a common technique, and shows up everywhere from designing little
languages to building complex data structures.

Polymorphic Variants

In addition to the ordinary variants we've seen so far, OCaml also
supports so-called polymorphic variants. As we'll
see, polymorphic variants are more flexible and syntactically more
lightweight than ordinary variants, but that extra power comes at a
cost.

Syntactically, polymorphic variants are distinguished from ordinary
variants by the leading backtick. And unlike ordinary variants,
polymorphic variants can be used without an explicit type
declaration:

As you can see, polymorphic variant types are inferred
automatically, and when we combine variants with different tags, the
compiler infers a new type that knows about all of those tags. Note that
in the preceding example, the tag name (e.g., `Int) matches the type name (int). This is a common convention in
OCaml.

The type system will complain if it sees incompatible uses of the
same tag:

# let five =`Int "five";;

val five : [> `Int of string ] = `Int "five"

# [three; four; five];;

Characters 14-18:
Error: This expression has type [> `Int of string ]
but an expression was expected of type
[> `Float of float | `Int of int ]
Types for tag `Int are incompatible

The > at the beginning of the
variant types above is critical because it marks the types as being open
to combination with other variant types. We can read the type [> `Int of string | `Float of float] as
describing a variant whose tags include `Int of
string and `Float of float,
but may include more tags as well. In other words, you can roughly
translate > to mean: "these tags or
more."

OCaml will in some cases infer a variant type with <, to indicate "these tags or less," as in
the following example:

The < is there because
is_positive has no way of dealing with
values that have tags other than `Float of
float or `Int of int.

We can think of these < and
> markers as indications of upper
and lower bounds on the tags involved. If the same set of tags are both an
upper and a lower bound, we end up with an exact
polymorphic variant type, which has neither marker. For example:

Here, the inferred type states that the tags can be no more than
`Float, `Int, and `Not_a_number, and must contain at least
`Float and `Int. As you can already start to see,
polymorphic variants can lead to fairly complex inferred types.

Example: Terminal Colors Redux

To see how to use polymorphic variants in practice, we'll return
to terminal colors. Imagine that we have a new terminal type that adds
yet more colors, say, by adding an alpha channel so you can specify
translucent colors. We could model this extended set of colors as
follows, using an ordinary variant:

We want to write a function extended_color_to_int, that works like
color_to_int for all of the old kinds
of colors, with new logic only for handling colors that include an alpha
channel. One might try to write such a function as follows.

The code looks reasonable enough, but it leads to a type error
because extended_color and color are in the compiler's view distinct and
unrelated types. The compiler doesn't, for example, recognize any
equality between the Basic tag in the
two types.

What we want to do is to share tags between two different variant
types, and polymorphic variants let us do this in a natural way. First,
let's rewrite basic_color_to_int and
color_to_int using polymorphic
variants. The translation here is pretty straightforward:

Now we can try writing extended_color_to_int. The key issue with this
code is that extended_color_to_int
needs to invoke color_to_int with a
narrower type, i.e., one that includes fewer tags. Written properly,
this narrowing can be done via a pattern match. In particular, in the
following code, the type of the variable color includes only the tags `Basic, `RGB, and `Gray, and not `RGBA:

The preceding code is more delicately balanced than one might
imagine. In particular, if we use a catch-all case instead of an
explicit enumeration of the cases, the type is no longer narrowed, and
so compilation fails:

Polymorphic Variants and Catch-all Cases

As we saw with the definition of is_positive, a match
statement can lead to the inference of an upper bound on a variant
type, limiting the possible tags to those that can be handled by the
match. If we add a catch-all case to our match
statement, we end up with a type with a lower bound:

Catch-all cases are error-prone even with ordinary variants, but
they are especially so with polymorphic variants. That's because you
have no way of bounding what tags your function might have to deal
with. Such code is particularly vulnerable to typos. For instance, if
code that uses is_positive_permissive passes in Float misspelled as Floot, the erroneous code will compile
without complaint:

With ordinary variants, such a typo would have been caught as an
unknown tag. As a general matter, one should be wary about mixing
catch-all cases and polymorphic variants.

Let's consider how we might turn our code into a proper library
with an implementation in an ml file
and an interface in a separate mli,
as we saw in Chapter 4, Files, Modules, and Programs. Let's start
with the mli:

In the preceding code, we did something funny to the definition of
extended_color_to_int that underlines
some of the downsides of polymorphic variants. In particular, we added
some special-case handling for the color gray, rather than using
color_to_int. Unfortunately, we
misspelled Gray as Grey. This is exactly the kind of error that
the compiler would catch with ordinary variants, but with polymorphic
variants, this compiles without issue. All that happened was that the
compiler inferred a wider type for extended_color_to_int, which happens to be
compatible with the narrower type that was listed in the mli.

If we add an explicit type annotation to the code itself (rather
than just in the mli), then the
compiler has enough information to warn us:

Once we have type definitions at our disposal, we can revisit the
question of how we write the pattern match that narrows the type. In
particular, we can explicitly use the type name as part of the pattern
match, by prefixing it with a #:

This is useful when you want to narrow down to a type whose
definition is long, and you don't want the verbosity of writing the tags
down explicitly in the match.

When to Use Polymorphic Variants

At first glance, polymorphic variants look like a strict
improvement over ordinary variants. You can do everything that ordinary
variants can do, plus it's more flexible and more concise. What's not to
like?

In reality, regular variants are the more pragmatic choice most of
the time. That's because the flexibility of polymorphic variants comes
at a price. Here are some of the downsides:

Complexity

As we've seen, the typing rules for polymorphic variants are
a lot more complicated than they are for regular variants. This
means that heavy use of polymorphic variants can leave you
scratching your head trying to figure out why a given piece of
code did or didn't compile. It can also lead to absurdly long and
hard to decode error messages. Indeed, concision at the value
level is often balanced out by more verbosity at the type
level.

Error-finding

Polymorphic variants are type-safe, but the typing
discipline that they impose is, by dint of its flexibility, less
likely to catch bugs in your program.

Efficiency

This isn't a huge effect, but polymorphic variants are
somewhat heavier than regular variants, and OCaml can't generate
code for matching on polymorphic variants that is quite as
efficient as what it generated for regular variants.

All that said, polymorphic variants are still a useful and
powerful feature, but it's worth understanding their limitations and how
to use them sensibly and modestly.

Probably the safest and most common use case for polymorphic
variants is where ordinary variants would be sufficient but are
syntactically too heavyweight. For example, you often want to create a
variant type for encoding the inputs or outputs to a function, where
it's not worth declaring a separate type for it. Polymorphic variants
are very useful here, and as long as there are type annotations that
constrain these to have explicit, exact types, this tends to work
well.

Variants are most problematic exactly where you take full
advantage of their power; in particular, when you take advantage of the
ability of polymorphic variant types to overlap in the tags they
support. This ties into OCaml's support for subtyping. As we'll discuss
further when we cover objects in Chapter 11, Objects, subtyping
brings in a lot of complexity, and most of the time, that's complexity
you want to avoid.