Embedded Domain Specific Languages in F# using Custom Operations

10 Oct 2014

Introduction

I've written before about F# and embedded domain specific languages.
This time we're going to explore a less widely known feature of F#, custom
operations within computation expressions, and see how they can help us to build
up a simple eDSL for describing NuGet packaging. All the code for this blog post
can be found on GitHub.

Custom Operations

Custom operations, not to be confused with custom
operators, were added to F# 3.0 and allow for custom keywords
and syntax to be defined within a computation expression. Custom operations are
the mechanism through which query expressions are implemented.

A simple NuGet eDSL

We'll break the problem down into three stages. First, we'll define a small set
of data structures that represents all the information we need in order to build
a NuGet package. Second, we'll define a NuGet computation expression, along with
custom operations that represents our eDSL, and use this to construct our data
structures. Thirdly, we'll pass this data structure to a separate function that
will actually perform the NuGet packaging operation according to the
specification defined by our eDSL.

Let's start by defining our data types. We'll need two main types, one to hold
information about the package itself (e.g. package ID, version) and another to hold
data about the packaging process (e.g. where to find the NuGet executable, where
to output the package). We'll also define some ancillary and convenience types
too.

// We're referencing the NuGet core package in order to get access to their// version data structure and parsing functionality.typeversion=NuGet.SemanticVersion// We'll just store the package ID and version for now. We'll expand this// shortlytypePackageDefinition={id:stringversion:version}// We'll also need to store information about where to find the nuget.exe, the// solution's root folder and an output folder. Most importantly, we'll need a// list of package definitions from which we'll generate the packages.typetypeNuGetDefinition={rootDirectory:stringtoolsDirectory:stringoutputDirectory:stringprojects:PackageDefinitionlist}

It might also be useful to define functions that provide a minimal set of
defaults for these data structures:

Given this extremely minimal definition of the data we need in order to create a
package we can now create an equally minimal DSL that allows us to build up these
data structures. The DSL will take the form of two computation expression
builders, each of which defines custom operations that represent the language of
our DSL.

The F# language specification requires that in order to define custom
operations the computation expression builder must also define the Yield
operation. The Yield function is invoked immediately upon entering the
computation expression so we're using it to setup the defaults. These default
values then get threaded through the remaining computation expression so that we
can make modifications to build up the package definition. This is how we'd
use our DSL so far.

At the moment this doesn't actually do anything useful, we'll need to add more
operations in order to collect all the data we need to build a useful NuGet
package.

One of the key features of NuGet is the ability to specify dependencies between
packages. Here's the definition of a DepdendencyDefinition structure that
we can use to define package dependencies.

//Just an example subset of versionstypeFrameworkVersion=|Net35|Net40|Net40ClientProfile|Net40Compact|Net40Full|Net45|Silverlight3|CustomofstringtypeDepedencyDefinition={name:string;version:version;frameworkVersion:FrameworkVersionoption;}

We can now extend our DSL to allow for package dependencies to be expressed.

We can go on adding more and more package dependencies in a similar way.
Constructing dependencies in this way, using record syntax, is not the most
friendly though. With a few helper functions we can clean it up to make it much
nicer to use, and also allow for optional arguments to be supplied or omitted
without having to use the full record syntax (see here for
an explanation of how static type constraints work).

// This is a declaration of an infix operator that will invoke an implicit// conversion from one type to another, as long as the impicit operator is// defined for either type.letinline(!>)(x:^a):^b=((^aor^b):(staticmemberop_Implicit:^a->^b)x)typeDepedencyDefinition={name:string;version:version;frameworkVersion:FrameworkVersionoption;}staticmemberprivatecreatenvf={name=n;version=v;frameworkVersion=f}staticmemberop_Implicit(x:string*version)=matchxwith|(n,v)->DepedencyDefinition.createnvNonestaticmemberop_Implicit(x:string*version*FrameworkVersion)=matchxwith|(n,v,f)->DepedencyDefinition.createnv(Somef)

We defined a new infix operator that will enable us to easily cast one type into
another using the !> syntax. We then declared a set of implicit conversion
for the DependencyDefinition type which accept a tuple of different lengths
and return us a DependencyDefinition structure. We can use this new syntax
and rewrite our original usage of dependency definitions like this

We can continue adding more and more features to our DSL that support all of the
packaging options that we want or need. Ultimately ending up with something that
allows a rich set of packaging options to be defined.

Once the packaging definition has been built up we can then pass it to another
function, that we'd have to write, to actually build the nuspec xml and call the
nuget.exe in order to do the packaging. A minimal, but complete, solution which
implements these ideas and also the actual task of building the packages can be
found on here.

Further Reading

You can get the full source from this article on GitHub. The rules
that govern exactly how custom operations get expanded are quite complex,
there's many additional options that we can use to tweak the way we express our
eDSL which I'll explore in future posts. As ever, the F# language
specification (section 6.3.10) is your friend.