Reactive Marble Testing with Functional Combinators in F#

I recently implemented marble testing in the F# reactive extensions project with functional combinators. In this blog post you'll discover why this functional approach is not only safer but more readable than any other object-oriented approach.

Stijn Moreels

29 November 2018

Marble Testing

Before we go down in the actual implementation, let’s quickly go over the key points of marbles in reactive testing.

What are “marbles”? When you have looked at the documentation of reactive programming packages, you often find those big diagrams explaining what the function does.

Because time in reactive programming is a very important factor, marble diagrams are in my perspective an excellent way of explaining time-dependent functions.

On the general Rx.NET website, you can find for example a marble diagram explaining the filter operator.

This way of representing reactive operators is the recommended way of explaining these operators because the “time”-factor is visually represented.

But what if we could use the same approach in our code? What if we could make our code just as self-explaining as these diagrams?

That’s what marble testing is all about.

Marble Syntax

This is not a post about marble testing itself, but rather how functional combinators can help with this. That being said, you should have a decent understanding of the rather “simple” syntax in which we express these marbles.

A “marble diagram” in reactive tests is just a series of characters representing “time”. Some characters have a special meaning:

- : (dash) means the “passing of time”, a single frame window.

| : (pipe) means the emit of a complete-event (in Rx.NET this is an OnCompleted emit)

# : (hashtag) means the emit of an error (in Rx.NET this is an OnError emit)

( ) : (parenthesis) means that we wrap one or more characters in a single emit. This means that ab results in OnNext(a); OnNext(b) and (ab) result in OnNext(ab).

any other character is assumed to be a value and is emitted in an OnNext emit.

So for example --a--b---#---(cd)-| means that

after 2 frames we get OnNext(a), then

after another 2 frames we get OnNext(b), then

after 3 frames we receive an OnError(), then

after 4 frames we get OnNext(cd), then

finally we receive the OnCompleted() after a single frame

Reactive Marble Testing the Filter Operator

Below you can see how we would write this with the implemented marble functionality. The first string represents the input notifications given to the filter operator. The second string represents the expected output.

Note that we filter on emits greater then 10, which is how the expected representation indeed leaves these emits out.

Note that the cold operator will create a Cold Observable for us for a given marble sequence.

Functional Combinators

Now that I’ve shown you a practical example of marble testing in F#, I can start explaining the actual implementation of this testing approach and how Functional Combinators, or more specifically, Functional Parser Combinators fits in.

F# has a port of the Haskell parser package Parsec which is called FParsec. What this package provides is a set of “combinators” to parse text in a functional manner. A combinator is in fact just a Higher-Order Function that is built from other combinators to create bigger/more-complex systems.
I like to think that combinators are in fact just a practical way of using function composition; only in an “extreme” way. But it’s still composition.

Ok, let’s get started. I will not explain the entire implementation but just some key points so you can get a feeling of how this works and can maybe use it for your next parsing assignment. If you want the full implementation, I suggest you take a look at the package itself: FSharp.Control.Reactive.

Parsing Frames

The first and most easy thing to parse, are the frames in the marble syntax. If you look back at the syntax you saw that a single frame is identified by the character - (dash). The FParsec package has several starting-point parsing functions:

pfloat parses a single float character (ex. 1.0)

pchar parses a single string character (ex. ‘c’)

pbool parses a boolean (‘true’ or ‘false’)

…

Now, what we need is the character parsing function. So to parse a single frame window we can use:

let pframe = pchar '-'

Voila, you just have parsed a single frame!

As you have noticed the marble syntax allows more than a single frame together, so we actually need to parse more then a single char.
Here’s our first encounter of a functional combinator. Instead of implementing something that will parse a string or a list of dashes, we re-use our first single frame window parser.

FParsec has a combinator called many which does exactly that. We parse while we don’t encounter a dash character anymore. This gives us

let pframes = many (pchar '-')

And by just that, you can parse one or more frame windows. It’s not only extremely short; it’s also very readable: “parse many dash char”.

Parsing Errors

Now that we can parse frames, lets see how we can parse error emits. Errors are represented by the hashtag symbol ‘#’. This is also easy for us. This can be be parsed with the same pchar combinator, resulting in: pchar '#'.

But now it gets interesting, we know that the error can be prefixed with any number of time frames and we need to know the exact number of frames so that we can create an Error emit for it. Luckily, the library has something to help us.

We need to parse the frames into an Error emit and discard the ‘#’ symbol itself. What we need is a combinator that parses two combinators but discards one of the two. FParsec has this and it’s written as: .>>. The dot stands for the part that you want to keep and the >> part is just the composition operator.

With this in mind and the previously created pframes combinator, we can now parse Errors with:

let perror = pframes .>> pchar '#'

Pretty neat!

Parsing Values

Because parsing values are a bit more complex that parsing errors, I deliberately chose to put this at the end. Values in the marble syntax are anything character except for parenthesis, hashtags and pipes. But any other character is parsed as a value. What’s different here is that we don’t know the exact value up front but we do know what it’s not. FParsec has something called noneOf which I used to solve this problem. Parsing a single value is now rather easy:

let pvalue = noneOf [ '('; ')'; '#'; '|' ]

That’s that!

The obvious step would also to include the prefixed time frames to this parser, but hang on. The marble syntax also specifies that we can wrap multiple characters as a single value if we write the characters between parenthesis. Funny enough, the combinator that we need is called: between which does exactly that: parsing a value between two other parsers:

let pvalues = between (pchar '(') (pchar ')') (many1 pvalue)

Note how I reused the already created pvalue combinator. What’s different though is that I used the many1 combinator instead of the many combinator. The former makes sure that we have at least a single character so we can safely assume that we always have at least a single character between our parenthesis.

Finally we need to choose between one or the other parser which can be done with the: <|> combinator resulting in our final value parsing combinator:

let pvalueOrValues = pvalue <|> pvalues

Note that if you want to test this, you have to map the result of the pvalue first to a String because the pvalues combinator returns that and the pvalue only returns a single character but for the sake of simplicity I omitted this.

Conclusion

I hope that I’ve convinced you that functional parsing combinators are indeed highly readable, community safe and easy maintainable. I’ve not covered the entire implementation but it can be found on GitHub. I hope you think of this post the next time you’ve got a parsing assignment.