However, this does not work. The definition of put is type correct (but dubious), but the definition of get is not type correct. And actually this makes sense: we are claiming that we can define Binary (Val a)for anya; but if the tag is 0, then that a can only be Int, and if the tag is 1, then that a can only be Double.

One option is to instead give a Binary (Some Val) instance with Some defined as

dataSome :: (*->*) ->*whereExists :: forall f x. f x ->Some f

That is often independently useful, but is a different goal: in such a case we are discovering type information when we deserialize. That’s not what we’re trying to achieve in this blog post; we want to write a Binary instance that can be used when we know from the context what the type must be.

Working, but inconvenient

The next thing we might try is to introduce Binary instances for the specific instantiations of that a type variable:

Note that there is no need to worry about any tags in the encoded bytestring; we always know the type. Although this works, it’s not very convenient; for example, we cannot define

encodeVal ::Val a ->ByteString
encodeVal = encode

because we don’t have a polymorphic instance Binary (Val a). Instead we’d have to define

encodeVal ::Binary (Val a) =>Val a ->ByteString
encodeVal = encode

but that’s annoying: we know that that a can only be Int or Double, and we have Binary instances for both of those cases. Can’t we do better?

Introducing RTTI

Although we know that a can only be Int or Double, we cannot take advantage of this information in the code. Haskell types are erased at compile time, and hence we cannot do any kind of pattern matching on them. The key to solving this problem then is to introduce some explicit runtime type information (RTTI).

We start by introducing a data family associating with each indexed datatype a corresponding datatype with RTTI:

data family RTTI (f :: k ->*) :: (k ->*)

For the example Val this runtime type information tells us whether we’re dealing with Int or Double:

We’re now almost done: the last thing we need to express is that if we know at the type level that we have some RTTI available, then we can serialize. For this purpose we introduce a type class that returns the RTTI:

and the good news is that this means that whenever we construct specific Vals we never have to construct the RTTI by hand; ghc’s type class resolution takes care of it for us.

Taking stock

Instead of writing

encodeVal ::Binary (Val a) =>Val a ->ByteString
encodeVal = encode

we can now write

encodeVal ::HasRTTIVal a =>Val a ->ByteString
encodeVal = encode

While it may seem we haven’t gained very much, HasRTTI is a much more fine-grained constraint than Binary; from HasRTTI we can derive Binary constraints, like we have done here, but also other constraints that rely on RTTI. So while we do still have to carry these RTTI constraints around, those are – ideally – the only constraints that we still need to carry around. Moreover, as we shall see a little bit further down, RTTI also scales nicely to composite type-level structures such as type-level lists.

Another example: heterogeneous lists

As a second—slightly more involved—example, lets consider heterogeneous lists or n-ary products:

As was the case for Val, we always statically know how long such a list is, so there should be no need to include any kind of length information in the encoded bytestring. Again, for serialization we don’t need to do anything very special:

The only minor complication here is that we need Binary instances for all the elements of the list; we guarantee this using the All type family (which is a minor generalization of the All type family explained in the same set of lecture notes linked above):

Serializing lists of Vals

This All Binary Val xs constraint however is unfortunate, because we know that all Vals can be deserialized! Fortunately, we can do better. The RTTI for the (:*) case (RttiNpCons) included RTTI for the elements of the list. We made no use of that above, but we can make use of that when giving a specialized instance for lists of Vals:

Note that this use of overlapping type classes instances is perfectly safe: the overlapping instance is fully compatible with the overlapped instance, so it doesn’t make a difference which one gets picked. The overlapped instance just allows us to be more economical with our constraints.

Here we can appreciate the choice of RTTI being a data family indexed by f; indeed the constraint HasRTTI f x in RttiNpCons is generic as possible. Concretely, decodeVals required only a single HasRTTI constraint, as promised above. It is this compositionality, along with the fact that we can derive many type classes from just having RTTI around, that gives this approach its strength.

Advanced example

To show how all this might work in a more advanced example, consider the following EDSL describing simple functions:

If you are new to EDSLs (embedded languages) in Haskell, you way wish to watch the Well-Typed talk Haskell for embedded domain-specific languages. However, hopefully the intent behind Fn is not too difficult to see: we have a datatype that describes functions: exponentiation, square root, integer modules, rounding, and function composition. The two type indices of Fn describe the function input and output types. A simple interpreter for Fn would be

In the remainder of this blog post we will consider how we can define a Binary instance for Fn. Compared to the previous examples, Fn poses two new challenges:

The type index does not uniquely determine which constructor is used; if the type is (Double, Double) then it could be Exp, Sqrt or indeed the composition of some functions.

Trickier still, Comp actually introduces an existential type: the type “in the middle” b. This means that when we serialize and deserialize we do need to include some type information in the encoded bytestring.

For our DSL of functions, we only have functions from Double to Double, from Int to Int, and from Double to Int (and this is closed under composition).

Serializing type information

The next question is: when we serialize a Comp constructor, how much information do we need to serialize about that existential type? To bring this into focus, let’s consider the type information we have when we are dealing with composition:

Whenever we are deserializing a Fn, if that Fn happens to be the composition of two other functions we know RTTI about the composition; but since the “type in the middle” is unknown, we have no information about that at all. So what do we need to store? Let’s start with serialization:

putRttiComp ::RTTIFn'(a,c) -> RttiComp '(a,c) ->Put

The first argument here is the RTTI about the composition as a whole, and sets the context. We can look at that context to determine what we need to output:

Let’s take a look at what’s going on here. When we know from the context that the composition has type Double -> Double, then we know that the types of both functions in the composition must also be Double -> Double, and hence we don’t need to output any type information. The same goes when the composition has type Int -> Int, although we need to work a bit harder to convince ghc in this case. However, when the composition has type Double -> Int then the first function might be Double -> Int and the second might be Int -> Int, or the first function might be Double -> Double and the second might be Double -> Int. Thus, we need to distinguish between these two cases (in principle a single bit would suffice).

Having gone through this thought process, deserialization is now easy: remember that we know the context (the RTTI for the composition):

Binary instance for Fn

The hard work is now mostly done. Although it is probably not essential, during serialization we can clarify the code by looking at the RTTI context to know which possibilities we need to consider at each type index. For example, if we are serializing a function of type Double -> Double, there are three possibilities (Exp, Sqrt, Comp). We did something similar in the previous section.

Deserialization proceeds along very similar lines; the only difficulty is that when we deserialize RTTI using getRttiComp we somehow need to reflect that to the type level; for this purpose we can provide a function

reflectRTTI ::RTTI f a -> (HasRTTI f a => b) -> b

It’s definition is beyond the scope of this blog post; refer to the source code on github instead. With this function in hand however deserialization is no longer difficult:

If desired, a specialized instance for HList Fn can be defined that relies only on RTTI, just like we did for Val (left as exercise for the reader).

Conclusion

Giving type class instances for GADTs, in particular for type classes that produce values of these GADTs (deserialization, translation from Java values, etc.) can be tricky. If not kept in check, this can result in a code base with a lot of unnecessarily complicated function signatures or frequent use of explicit computation of evidence of type class instances. By using run-time type information we can avoid this, keeping the code clean and allowing programmers to focus at the problems at hand rather than worry about type classes instances.

PS: Singletons

RTTI looks a lot like singletons, and indeed things can be set up in such a way that singletons would do the job. The key here is to define a new kind for the type indices; for example, instead of

In such a setup singletons can be used as RTTI. Which approach is preferable depends on questions such as are singletons already in use in the project, how much of their infrastructure can be reused, etc. A downside of using singletons rather than a more direct encoding using RTTI as I’ve presented it in this blog post is that using singletons probably means that some kind of type level decoding needs to be introduced (in this example, a type family U -> *); on the other side, having specific kinds for specific purposes may also clarify the code. Either way the main ideas are the same.