Crushing Haskell like a Tin Can

Generics and Protocol Buffers

May 20th, 2012

We use Protocol Buffers extensively, and from talking to some folks at BayHac’12 it may be time to revisit the state of protobuf in Haskell.

To be fair, the protocol-buffers package is great. It’s extremely full featured, well tested and I can’t complain about the performance. But when most parties involved are running Haskell, maintaining separate .proto files is more than just a chore. Properly integrating the hprotoc preprocessor into a build system has also proven to be a challenge primarily due to the n:m mapping of source files to target modules.

After spending a little time this evening hacking around, I’ve come up with an alternate solution that looks promising and doesn’t require external files or additional build tools. Though it’s far from a production effort, the type-level version of the code is available on Github for all your forking needs.

Note: GHC 7.2 or up is required for Generic support.

So what does it look like?

By defining a set of types that allow tagging a record field with a field number…
<div class=’bogus-wrapper’><figcaption></figcaption><div class=”highlight”><table><tr><td class=”gutter”><pre class=”line-numbers”>123
</pre></td><td class=’code’><pre>newtypeRequired(n::Nat)t=RequiredtnewtypeOptional(n::Nat)t=OptionaltnewtypePacked(n::Nat)t=Packedt</pre></td></tr></table></div></div>

and a few more to override the default base-128 varint encoding…
<div class=’bogus-wrapper’><figcaption></figcaption><div class=”highlight”><table><tr><td class=”gutter”><pre class=”line-numbers”>12
</pre></td><td class=’code’><pre>newtypeFixedt=FixedtnewtypeSignedt=Signedt</pre></td></tr></table></div></div>

… should give you enough rope to write regular Haskell records that are efficiently (de)serialized with very little fuss. Create an annotated record, derive a Generic instance and you’re done.