The Test

We’re using with to destructure the results of our calls to do_foo/0 and do_bar/0 function calls. Next, we’re asserting that foo should equal bar.

If do_foo/0 or do_bar/0 return anything other than an :ok tuple, we’d expect our pattern match to fail, causing our test to fail. On running our test, we see that it passes. Our do_foo/0 and do_bar/0 functions must be working as expected!

The False Positive

Our do_bar/0 is returning an :error tuple, not the :ok tuple our test is expecting, but our test is still passing. What’s going on here?

I find myself producing lots of ranging content from books and articles, like the one you're reading now, to open source software projects. If you like what I'm doing, nothing shows your support more than signing up for my mailing list.

It’s easy to forget (at least for me, apparently) that when a with expression fails a pattern match, it doesn’t throw an error. Instead, it immediately returns the unmatched value. So in our test, our with expression is returning the unmatched {:error,:asdf} tuple without ever executing its do block and skipping our assertion entirely.

Because our assertion is never given a chance to fail, our test passes!

The Fix

The fix for this broken test is simple once we recognize what the problem is. We’re expecting our assignments to throw errors if they fail to match. One surefire way to accomplish that is to use assignments rather than a with expression.

Now, the :error tuple returned by our do_bar/0 function will fail to match with our :ok tuple, and the test will fail. Not only that, but we’ve also managed to simplify our test in the process of fixing it.

Success!

The Better Fix

After posting the above fix in response to my original tweet, Michał Muskała replied with a fantastic tip to improve the error messaging of the failing test.

While we’re still given all of the same information about the failure, it’s presented in a way that’s easier to read and internalize, leading to a quicker understanding of how and why our test is failing.

I’ll be sure to incorporate that tip into my tests from now on. Thanks Michał!

Modeling Formulas with Recursive Discriminators
– I ran into an interesting problem recently where I needed to model a nested set of either/or sub-schemas. With some creative thinking and a healthy dose of recursion, Mongoose's discriminator feature turned out to be just the tool for the job.

Generating Test Fixtures with Wireshark
– Wireshark can be an invaluable tool for testing the parsing and serializing of a well-known binary protocol. Check out how we can use binary fixtures exported from Wireshark to test our Elixir-based Bitcoin protocol parser and serializer.

Feedback from my readers is what motivates me to keep working in public. Feel free to reach out, or even sign up for my mailing list if you'd like to stay up to date with what I'm working on.