During the previous Dutch Scala Enthusiasts (DUSE) meeting some people mentioned the fuss about the Dynamic trait. Since ‘method missing’ was one of the features I enjoyed in Ruby and Groovy I decided to dig around and create some examples.

But, first of all: the example code in this post works with the latest RC (2.9-RC2) release… and only when the ‘-Xexperimental’ flag is passed to the compiler. Previous versions of 2.9 actually used different names, so this stuff might change while you’re reading this post.

Basically the using the Dynamic trait you can supply an exit-point to the compiler to invoke when a non-existing method is being called on a given object. Some interesting posts have been written about this feature in Ruby. A good overview of what to use the pattern for was written by Ola Bini a while back.

The following patterns caught my imagination in the past:

Encoding parameters in the method name (heavily used by Active Record -> entity.find_by_name())

Builders i.e. creating structured data without quote noise. There are various XML and JSON builders in the Groovy and Ruby world using this approach

Let’s see some examples of the above using the Dynamic trait in scala:

Encoding parameters

ActiveRecord is one of the first examples I came across which really showed me the power of encoding parameters in the method name. It let’s you write queries for your entities as methods:

[code]
User.find_by_name('peter')
User.find_by_id(1234)
[/code]

The column name is extracted from the method and the call is dispatched to a generic find method.

Using the Dynamic trait in Scala we can do something very similar to the above:

Reflectively a suitable target is searched and invoked on the ParameterInMethodNameExample instance

Sure, this is not ActiveRecord yet… but demonstrates the possibilities very well. For something which is called a statically typed language this feels (is?) _very_ dynamic.

Builders

I’ve created a very simple builder to demonstrate how that could work:

applyDynamic just keeps appending new path elements to their parent, and using a recursive toString this prints the actual path. Not very useful, but illustrates how something more elaborate could work.

Do we want this?

A lot of people are/will be concerned about the disadvantages features like this bring to a language. I do however think it is a nice piece op equipment in the toolbox. And like other features (operator overloading?): handle with care… powerful tools can do a lot of damage when used incorrectly.

Today I was playing around with some code to show people how Scala traits (more specifically stackable traits) make it much easier to decompose your ‘problem’ into generic and reusable building blocks.

One of the issues we recently looked at was retrying a query which failed for some reason; which might not sound like something you’ed want to retry.. but believe me; in some cases it makes sense.

So, say we have a very simple repository definition to access a database:

We can now ‘implement’ a couple of dummy repositories to play with:

As you can see we have three different repositories, with their own unique characteristics. Calling the first two repositories would result in exceptions. The second one does however work when we call retry the call a couple of time. Now, enter the Retrying trait:

As you probably guess from the name: this trait will add retrying functionality to the class which it is mixed in to. So, how does it work:

The abstract modifier on the override of the withConnection allows us to change the behavior of the method we’re overriding

The super.withConnection call in the retry function is still referring to the actual implementation of the class the trait is mixed in to (well, or the first override in the inheritance chain)

We can mix this trait into a repository in the class definition, or… more interesting… at instantiation time:

The above code will print something like:

number of tries left: 3
number of tries left: 2
number of tries left: 1
number of tries left: 0
failed result: None
number of tries left: 3
number of tries left: 2
number of tries left: 1
success result: Some(should fail first, succeed later)
number of tries left: 3
success result: Some(should not fail result)

Surely this is only the tip of the iceberg. But as you can see, this is a very powerful mechanism to create a very modular set of very specialized components; each with their very specific purpose. High cohesion… low coupling!

Ow… and yes I notice my retry function is not tail recursive; still working on that…

No, writing executable scala shell scripts is not rocket science. I know. I noticed however that people don’t often use scala in this fashion because it didn’t occur to them.

Have a short look at the following utility I wrote the calculate the scrabble® score for every line in the given input file. The ‘magic’ is in the first lines (which I learnedhere) of the file, which passes the filename and arguments to a scala command (which must be in you path to work for this).

I you put this code in a file and make it executable (ie. chmod a+x <nameOfScriptFile>) you can execute it like

Scalatest offers some very elegant ways to layout your tests. After using Selenium2 at work I started thinking how I could leverage the BDD-like goodness in combination with Selenium2. The combination proves to be very useful.

Some basicsSelenium2 offers a couple of drivers which allow your code to spawn webbrowsers (or a virtual browser) and control and monitor them.

In Scala starting a Firefox instace and opening a url would look like:

After opening the page you can use the driver to browse the actual Dom of the response. Methods for this include stuff like findElementsByTagName, ClassName, Id and XPath. It even allows you to make a PNG image of the page.

Bind pages to classes
Next to the above methods the driver can ‘bind’ values in the page to bean properties. I discovered that Selenium has no problems with populating annotated Scala classes:

combined with FeatureSpec
FeatureSpecs allow you to describe scenarios of features, and has options to create ‘pending’ tests; very useful for test driven development. If we combine Selenium with this type of test we get something like this:

then("the url for that category should be openend")
driver.getCurrentUrl should endWith("/category/scala/")

then("all posts should have the selected category in their metadata")
val categoryPage = PageFactory.initElements(driver, classOf[BlogPage])
categoryPage.verifyPostCategories("scala") should equal(true)
}

}
}

[/code]

This test will:

Start Firefox

Open the frontpage of this blog

Select ‘scala’ in the category box in the menu on the right

Verify the opened url

Verify that all posts in the category page have the selected category in their metadata

If you run the test, scalatest will create some very nice output:

[code]
Feature: A weblog should have a archive based on post categories
As a visitor
I want to be able to select a category
Scenario: we select the 'scala' category to see posts about Scala
Given we open the frontpage page
Then we should see a widget containing categories
Given we select 'scala' using the select widget
Then the url for that category should be openend
Then all posts should have the selected category in their metadata
[/code]

Keen observers might have noticed some helper methods on the BlogPage class, like ‘selectCategoryName’ and ‘verifyPostCategories’. Using the collection conversions from Scala 2.8 we don’t even have to wrap the Selenium API to make them look nice:

As you can see translating stories into real integration tests is quite simple; and it should be possible to run the tests in different browsers (Chrome, IE) as well. And since you won’t be writing production code it could be a very nice way to give scala into your project!

After the SQL example I also showed some of the work I did on a custom templating language I wrote as a experiment for Ebay/Markplaats for which I wrote the parser and interpreter in Scala. Maybe I can give some more details on that in the future.

Last Tuesday I gave a ‘introduction to Scala’ presentation during the monthly ‘Tech Tuesday’ at Ebay/Marktplaats. After going through the slides I did some live test driven development. I think I managed to get some more people interested!

Last week was my first week at my new employers office. At eBay / Marktplaats I’ll be working on the upcoming version of Marktplaats and various applications/services part of the platform around it.

You can either bring your own hardware or use a corporate laptop. Since I don’t actually own a laptop I went for the corporate machine. I got a (huge) Dell Latitude D830. It contains a T7700 core duo 2.4gz, 4gb of memory and a NVidia Quadro NVS140M (business version of the NVIDIA GeForce 8400M G).

I installed the 64- bit version Ubuntu 9.10 (karmic) from a bootable USB stick during my first day. Installation went smooth. Getting X to recognize all display configurations took some work and is still far from perfect: stuff like detaching a display/docking system and attaching another display doesn’t really work without manual intervention. I setup VMWare and a WinXP image to be able to access all corporate tools and test in IE.

What I like:

Compiz has some nice tricks for windows selection. alt-shift-up for window selection and some exposé-like features which I mapped to F9 and F10 like I was used to on Osx.

Gnome Do is a workable replacement for Quicksilver

Suspending/Hibernating actually works well and configuration is far more flexible then it is on OSX

apt is so much better than macports/fink

I like gnome virtual desktops better dan spaces on OSX

What I (still) don’t like (compared to OSX):

Copying text between applications is still not consistent (at least for me)

Input dialogs don’t get focus all the time

Reconfiguration of X is flaky

Still looking for a workable replacement for Omnigraffle, using Dia for the moment

I miss Mail.app/Addressbook/iCal

Some stuff is harder to setup then it should; skype for instance

Various hardware related aspects: multitouch trackpad being the most important one

But overall the experience is good. I got productive within a day; gave a presentation and used the corporate tools through VMWare. I’ll try and see if I can improve switching X configs, which would really improve the overall experience.

While writing my internal SQL DSL example I stumbled upon this blogpost about a Type-safe Builder Pattern in Scala a couple of times. One of the issues I have with that code is the unnecessary use of mutable state in the example. Which are probably there to demonstrate the validation of the result at the end of the article.

The blogpost concludes with some complex implicit solution to have the builder only create valid configurations, but (IMHO) fails to address the fact that it is still possible to create OrderOfScotch variants which are invalid through its’ constructor.

With Scala 2.8 it is possible to vastly simplify the example by using the the copy method on the case classes and default arguments:

When using an incorrect configuration (in the example implementation a double scotch, in a short glass) creating the Order throws an exception. And since we in effect only use the constructor to create the Order this works in any case.

As you can see the amount of boilerplate code was reduced dramatically, we’ve got rid of all mutable state and moved validation of the Order to a (IMHO) better location!

One ‘drawback’ of my solution is of course that it only works on runtime; whereas the solution presented in the catches incorrect configurations created by the builder at compiletime… which was the entire point of that blogpost. I’ll try and see if I can factor that into a more compact form as well lator on!

Following up on my previous post on writing an internal SQL-like DSL for Scala I decided to bite the bullet and implement a ‘real’ parser for a subset of the SQL language to create the object tree from a SQL-like string. Since I didn’t have any experience in Parser/Combinators this proved to be quite an interesting exercise.

At this point the specs for the following queries succeed and produce valid results when I render SQL from the object graph:

[code]
select name from users order by name asc
select name from users where name = "peter"
select age from users where age = 30
select name from users where name = "peter" and age = 30
select name from users where age = 20 or age = 30
select name from users where name = "peter" and age = 20 or age = 30
select name,age from users where name = "peter" and (active = true or age = 30)
[/code]

It matches a string which starts with order by, then an identifier and ends with asc or desc. The tilde (~) depicts a separater, and the greater then (>) after the tilde is used to drop the fields on the left side of the operator; just to have less repetition in the match algorithm to bind the value into objects which will form the AST.

And/Or en parentheses
Implementing and/or in the where clause proved to be more difficult then I expected; mainly due to the precedence rules when using parentheses. I asked around on stackoverflow and read how this is done in this great article by Jim McBeath.

Since the Query object already accepts optional objects as where and order clause binding is straightforward.

Conclusion
Being fairly new to this game I had some trouble finding out how I was supposed to approach some of the problems I faced; but managed to overcome them quickly. There is a lot of documentation on various blogs and books. Writing a parser in Java, Groovy or Ruby would probably have taken me more time and would probably have resulted in far more code.

The implicit conversions allow tuples to be converted to typed case classes. With the above we can write Scala code which resembles SQL. I Wrote some tests (using ScalaTest) which demonstrate how it works. Example inputs for my tests include:

Apart from the parentheses (probably I’ll figure out how to get rid of some more of them one day) the two look very similar. But the first one creates a typesafe object graph which can be rendered/validated/manipulated in various ways!

The example could be extended to allow ‘greater/smaller then’ clauses (using operator overloading?) or joins in the from clause; feel free to clone my repository and do so!

About

Welcome to the weblog of Peter Maas. Here you'll find various posts related to stuff I like (like my kids and espresso) and stuff I do (like developing software).

Disclaimer

I am a very opinionated software engineer. Well I'm a very opinionated person in general. I work for eBay/Marktplaats.nl, but everything written here is my own crazy ideas and not my employer's. This text might have some similarities to the disclaimer found at fupeg.blogspot.com.