Software engineering

Scala Macros: Part I - Def Macros

Jan23rd, 20164:21 am

I. Introduction. What are scala macros?

Macros in scala allow you modify existing, or even generate new code before it gets compiled. They are used under the
hood of many known libraries in scala ecosystem like shapeless, slick or play framework.
They come in different flavors, one of them are: def macro and macro annotations.

In this blog post I will give you example of how you could easily start writing your own def macro. In Part 2 I will give
you more practical usage of macro annotations and in Part 3 we will build intellij idea plugin to support our macros.

II. How macros work?

You might think about scala macros like about framework/api for generating/modifying Abstract Syntax Trees(AST).

A) What is AST?

Abstract syntax tree is core data structure used in compilers. Early stage of compiling code is lexical analysis.
What it does is checking the sequence of code elements and it assigns them to tokens. Next stage is parsing code.
What parsing does is checking sequence of tokens in terms of grammar and builds some data structure base on parsed code.
This data structure is called parse tree and it includes each token found in parsed string. AST is similar but without
redundant information. Let’s look at this simple string:

1

1 + (2+3)

after lexical analysis this string could look like this:

1

int(1) '+' '(' int(2) '+' int(3) ')'

Now compiler need to have some structure which will give him an information about grammar relationships between those
tokens. Without redundant information like parenthesis etc. AST will provide it, and it could look like this:

Or maybe this ‘tree like’ structure, would be more intuitional:

After creating AST, it is further passed to compiler which makes use of it. What macro does is it “replaces” AST written
by developer with AST produced by macro itself, and then compilation goes further. So we are not resigning from any advantages
of static typed, compiled language assets.

Macros must be defined in other compilation phase(in other sbt project in our case).
I’ve created new project called
macros_implementations, and make our main project dependent on it. By the way this line:

build.sbt

1

lazyvalmacros_implementations=project

is also scala macro. According to sbt docs:
“The name of the val is used as the project ID and the name of the base directory of the project”
So we have 2 projects. Main one and dependent one called macros_implementations. To use macros we will also need compiler plugin.
called macro paradise.

IV. Example no.1: def macros

Look at Main.scala. I’ve placed there example of macros usage.

Main.scala

123

objectMainextendsApp{Welcome.isEvenLog(12)}

From “outside” developer cannot specify if method is implemented with scala macros or not. Invoking of def macro is the
same as any other method. What this macro does, it println some message depending on argument is even or odd.
In main project directory run:

1

sbt run

to see result of this method. So let’s look closer at the implementation.

Next, we need some api to modify AST, we would like to have some typechecker, some error logger etc:

1

importscala.reflect.macros.blackbox.Context

Macros in scala comes in 2 flavors: blackbox and whitebox. What is important: use blackbox ;). Talking seriously,
if macro faithfully follows its type signatures, and its implementation is not necessary to understand its behaviour
then we call that macro blackbox. In other case, in whitebox macro type signature is only an approximation.

To specify macro implementation we use keyword macro followed by implementation method name.

This implementation method takes 2 parameters:
first one is mentioned earlier Context, and second is parameter as element of Tree. Tree is node type of AST. Returning type
is also Tree.

Next thing is importing context universe which provide API for modifying AST.

And the main part of out implementation: building a Tree. This q”“ interpolator is called quasiquotes. What it does provides
developer an easier way to create/modify AST. I’ve said ‘easier’, so there should be also harder way. Let’s modify a little
bit body of our macro to see what q”“ produces in our case.

If(Apply(Select(Apply(Select(Literal(Constant(12)),TermName("$percent")),List(Literal(Constant(2)))),TermName("$eq$eq")),List(Literal(Constant(0)))),Apply(Ident(TermName("println")),List(Apply(Select(Apply(Select(Apply(Select(Literal(Constant("12")),TermName("$plus")),List(Literal(Constant("= ")))),TermName("$plus")),List(Select(Literal(Constant(12)),TermName("toString")))),TermName("$plus")),List(Literal(Constant(" and it is even")))))),Apply(Ident(TermName("println")),List(Apply(Select(Apply(Select(Apply(Select(Literal(Constant("12")),TermName("$plus")),List(Literal(Constant("= ")))),TermName("$plus")),List(Select(Literal(Constant(12)),TermName("toString")))),TermName("$plus")),List(Literal(Constant(" and it is odd")))))))

I hope that you see now, what quasiquotes give you ;) Each of Those If, Apply, Select objects are Tree subclasses, and
you can build your AST using them instead of quasiquotes, but it would be frustrating to do this, when you can use q”“ and
write human readable code.

To understand more what is going on in this example let’s modify our code a little bit. In Main.scala write this code:

Main.scala

12345

objectMainextendsApp{valx=2valy=3Welcome.isEvenLog(x+y)...

And to see generated code change showRaw to showCode in Welcome.scala:

You should see generated code in console. Without redundant syntax, generated code looks like the following:

12345

if((x+y)%2==0){println((x+y).toString+" is even")}else{println((x+y).toString+" is odd")}

So you maybe see now better that this (number: c.Tree) parameter of #isEvenLogImplementation is exactly only tree node.
Not Int, it is just some token, you might say, which is placed in places where you specify it. Of course this is some kind
of hidden bug(multiple evaluation), because you can imagine what will happen if you provide non deterministic argument,
like random integer:

Main.scala

1234

objectMainextendsApp{valx=2Welcome.isEvenLog(x+Random.nextInt())...

Result will look like the following:

12345

if((x+Random.nextInt())%2==0){println((x+Random.nextInt()).toString+" is even")}else{println((x+Random.nextInt()).toString+" is odd")}

And resulted print statement often will be as smart as following:

1

17 is even

The simplest fix of this issue is to explicitly evaluate input arguments first, so assign them to val, and use
those vals further in generating AST:

V. Summary of part 1

I hope that this simple example will encourage you to experiment with def macros. If you have been using play framework,
then probably you have been parsing json too, so look at source of Play json api
and check how macros are used to reduce boilerplate in play framework. There was a post about this json api, available
here. Few useful links, where you can find more extensive/formal answers
to the topic: