Custom validity and validity-based testing in
Haskell

Values of custom types usually have invariants imposed upon them. In this
post I motivate and announce the validity,
genvalidity and genvalidity-hspec libraries that
have just come out.

Contrary to what we might like to think, an absence of compilation errors
does not imply correctly functioning code. In some cases we expect some
invariants to hold about our data, but we don’t necessarily check them
rigorously.

This is a situation in which the typesystem may be expensive to use to
guarantee correctness. That’s why we will use testing instead.

A running example

I will take a contrived example to keep the blogpost short. Assume we have
the following context:

Genvalidity

Now that we have this concept of Validity, we can start
writing tests using it. We will ignore the tests which assert that the output
is a valid prime factorisation for now. We can then write the following tests
with respect to validity:

This is quite a mouthful, isn’t it? There are also some non-stylistic
problems:

someGenerator `suchThat` (not . isValid) runs
someGenerator and retries as long as isValid is
satisfied. For types that are mostly valid, this can take a long time and
will slow down testing significantly.

Generators like GreaterThanOne <$> arbitrary become
quite large (in code) for larger structures. Ideally we would only write
them once.

genInvalid Has a default implementation, but when generating
GreaterThanOnes, on average 50% of all generations have to be
retried at least once. We can specialize this implementation to run faster by
using an absolute value function:

Standard tests

We can use these new toys to write tests as described above, but we can
also use some of the standard tests that are available via
genvalidity-hspec. For example, the above tests can be rewritten
as follows:

Because we have now written a custom implementation of
genValid for GreaterThanOne, we should also add the
validitySpec:

validitySpec (Proxy ::ProxyGreaterThanOne)

This will ensure that genValid and genInvalid
keep working as intended. However, it cannot check that all possible
valid(/invalid) values can still be generated by
genValid(/genInvalid), so be careful and check that
yourself.