Debasish Ghosh on DSLs and Akka

Recorded at:

Bio Debasish Ghosh is the CTO of Anshinsoft Corp, where he specializes in leading the delivery of enterprise-scale solutions. Debasish is the author of "DSLs in Action" (http://www.manning.com/ghosh/), and blogs about DSLs, Scala, Functional Programming and much more at http://debasishg.blogspot.com and tweets at http://twitter.com/debasishg

QCon is a conference that is organized by the community, for the community.The result is a high quality conference experience where a tremendous amount of attention and investment has gone into having the best content on the most important topics presented by the leaders in our community.QCon is designed with the technical depth and enterprise focus of interest to technical team leads, architects, and project managers.

I am Debasish Ghosh, I work in Anshinsoft. It started as a startup in Silicon Valley. Our main development center is in Calcutta, India and we specialize in developing and delivering back office systems for capital markets: securities, trading, derivatives, financials and things like that. Actually I am the CTO of Anshinsoft? and besides my day job I am quite passionate about programming, particularly polyglot programming and these days I mostly program with Scala, and apart from Scala I also use Clojure, Erlang and some amount of Haskell in my projects.

The main motivation of writing a DSL book stemmed from my interest in polyglot programming. When I started learning more than one language and tried to apply all of those languages in my projects, what I found is that it would be ideal if I can model my solution domain more in line with what the problems are. So if the problem demands a language for which I can use a different language to model the solution domain, then it becomes much more clear to the implementer as well as to the user. So that was the main motivation for my being passionate about DSLs and then Manning got in touch with me. The chief editor, got in touch with me and then things rolled and we decided to do write a book on DSLs.

Actually both. What I try to do is I try to address both the user level perspective and the implementation level perspective of DSLs. I start of with a user level one, how to use a DSL, what are the things to look for when you select a DSL and then in the second part of the book I delve into the details of implementing DSLs. When I talk about implementing DSLs, obviously DSL is intimately directed with polyglot programming because you can’t model all problems using one language. So I decided to use multiple languages mainly to focus on the idea of thinking in different languages, because in order to model the problem you need to you need to think in terms of the issues, all the issues which address the domain and then finally when you come up with the solution model, the solution model should be implemented in a language which offers you the best abstraction level to model the problem.

I discuss internal DSLs as well as external DSLs. External DSL discussion is mostly focused on Scala because Scala is one of the languages which offers powerful functional parser combinators, which I discuss, and other thing which I discuss about external DSLs is the excellent framework which eclipses Xtext. Xtext I found to be a very good framework for developing and maintaining the entire life cycle of external DSLs. While designing internal DSLs I start off with the very familiar builder pattern of Java and then I show it how it can be improved with Groovy and finally I go into various snippets all of real world usages in terms of Scala, Groovy, Clojure and Ruby.

One of the things which I tried to highlight in the book is that I didn’t use multiple domains, I used one single domain to explain all the features of DSL design. That is primarily because I wanted to go into the depth of a domain to discuss the intricacies because designing very trivial DSLs, modeling state machines and things like that tend to be quite easy in every language, but when I delve deep into the complexities of a specific domain then the actual complex business rules come up. So my approach was to go into those business rules, give the user an understanding of the background of what these business rules are supposed to do and then try to implement it in multiple languages.

The implementations will look radically different because when you are using multiple languages, say you are using a language like Ruby, you tend to use the power of runtime meta programming, but the same thing when you model in terms of a LISP-based language, Clojure, you tend to model it in the form of syntactic macros which are compile time programming. Similarly if you model the same business tool in a language like Scala or Haskell, you tend to explore the type system, because the type system is so expressive, that you can express a lot of business logic within the type system itself, so your business logic code for which you need to write the test case, that comes down considerably.

Personally I am a big fan of static typing, so that makes Scala and Haskell two of my favorite languages, but personally, right now I am mostly using Scala, so yes, static typing is the thing which I like a lot.

Xtext gives you a graphical user interface, an IDE kind of thing to design your external DSLs. You write the grammars for your DSL in a similar format in which you'd write with ANTLR or any other tool like that and then Xtext gives you the framework to design an editor kind of thing where you get the advantages of syntactic highlighting and things like that. And it stores the DSL in the form of some meta data so that it can manage the entire life cycle of the DSL. You can change the DSL, you can maintain it, that is the main part of it that you get a very user friendly interface to managing the life cycle of your external DSL, because external DSL on the face of it looks quite difficult, many people view it as designing a complete language upfront.

It may be true for very complicated external DSLs, but for most of the external DSLs it may not be that complicated, and particularly with abstractions like parser combinators which are there in various languages, like Scala has one, the upcoming NewSpeak has very powerful combinators. So with these kinds of abstractions in your way designing external DSLs is becoming much more easy and user friendly.

Actually Scala parser combinators they are the functional abstractions, functional abstractions with which you can design your own DSL. So the advantage is you feel like writing a grammar, but in essence every snippet of that grammar is actually a parser and it uses monads to join all those parsers, so the ultimate big parser which you get is actually a monadic bind, it’s actually built through monadic binds of multiple smaller parsers. So that is the main advantage of it. If you have a clear idea of how your language looks like and you can model it in terms of a BNF kind of syntax then you can literally transfer that BNF format into parser combinators of Scala.

Let me be clear. I am not a category theorist, or I don’t have much idea, much knowledge about the theoretical underpinnings of a monad, from the category theory point of view. I am a practitioner. I love programming and I view monad as one of those powerful abstractions which help me compose smaller abstractions to give rise to bigger ones. And monad is not the only one. There are applicative functors also. In fact, applicative functors in some cases they come as having more power than monads because applicative functors compose more easily than monads. They are less restrictive than monads. It’s really that entire type class hierarchy which started with Haskell possibly and in Scala we have the scalaz library which gives us the power of these monads.

So it’s all related to composition of functions, function composition and binding that makes this functional abstraction so useful. In fact it’s one of the tenets of functional programming that functions compose mathematically. Unless you have any side effects within it you can compose functions and compose your abstractions in a pure way. You don’t have any side effects, so you have all those things of referential transparency, trivial parallelizability, all those things come along free with pure functional programming.

It depends on the way you compose, because all these abstractions that started possibly with the typed languages, so the types have to agree, you just cannot compose any two functions together. The types have to agree and depending of the agreement of the types you can either do a Kleisli composition or you do a monadic bind, or you can compose functors like that. So in effect yes, it’s composition in a way.

It’s not true, it’s a myth. The problem that DSLs solve is it improves the communication between the various stakeholders of a project. The business user is not a programmer, but if I can present a business rule to him in a form which he can understand then he can verify the correctness of the business rule which I am implementing, before the program goes out of the labs. So this is the most important value addition that DSLs bring to our ecosystem. With a DSL a business user can understand the business rule because none of the developers understand the business rule as good as the business user. So when I can convince a business user that my implementation is correct, in that case I can also be sure about the correctness of the implementation. So that I think is the main value addition of a DSL. It improves the communication.

So this is one level of collaboration between the business user and the programmer. Now when my software is ready and when my software is released then I need to have a team of programmers who need to maintain that software. So the persons who will be maintaining it they also need to understand the logic of the program. So if the business rule is very explicit in the way I express them, in that case the person who is maintaining the program it also becomes easy for him. So it’s the second level of collaboration programmer to programmer, so DSL help you do that, DSL help you collaborate on two fronts, programmer to business user and between programmer to programmer. So that I think, is the main value of DSLs.

I’ll give you an example of a real world use case. In one of my projects what I did was that we designed a DSL for a complicated business role and then I had my team of business users write test cases based on that DSL, that they could do because the scripting language was also a DSL and since they understand the business they could design a very exhaustive set of test cases using the DSLs which we designed. So that is one of the real world use cases of how much, at what length you can go with a DSL with your business users. You can’t expect them to write them big programs using DSLs, because one of the problems with internal DSL is the exception handling.

You design a very friendly DSL in Scala or Ruby or Groovy, but once you have an exception you may get a stacktrace or some kind of exceptions which the business users may not understand. And it’s still I think an open problem to design proper level of exception handling using DSLs.

Problematic, because in those cases you may have to use those foreign function interfaces and things like that which can turn out to be messy. And even on the same language platform, the same runtime, you need to choose the best way of integrating, because when we are on the JVM since Java 6, we have this script engine. Script engine offers you one level of integration, but there may be a case that the languages that you are using may have a better way of integrating. So always prefer the one which is the most natural mode of integration between the two languages.

It’s mainly securities trading applications. My book also contains real world examples from this domain modeling a securities trading back office, all the securities trades, settlements, accounting, managing the books, things like that. So this is the main domain which I work on. And almost if you read my blogs then you will find that most of this domain only.

Akka is the next generation middleware. It offers you actors as the primary form of interaction between systems, event based interactions, asynchronous, all of these are recipes for high scalability and things like that. So the primary value addition of Akka is the actor implementation, a robust actor implementation on the JVM.

They don’t map one to one with threads, obviously, because in that case it will not scale. So every actor in Akka comes with a dispatcher and you can customize your dispatcher implementation based on your requirements. You can use a thread pool, you can use executors and things like that. So there are lots of ways you can customize your dispatcher to suit to your requirements.

Yes, actually the idea is to bring that whole of Erlang goodness on to the JVM because along with the actors which Erlang runtime gives you is the concept of fault tolerance, supervisors, all of these things have been implemented in Akka. In fact you can think of Akka as having almost all the functionality of the Erlang runtime where you can manage your actors, you can follow the "let is crash" philosophy. Actors can crash and it will be reported to some other actors to which it’s linked. You can design supervisors and things like that.

Multiple JVMs hasn’t yet been done, I think, but it’s in the labs, we have plans of doing that. Currently what you can have is you have the concept of remote actors. It’s asynchronous, you can pass messages across JVMs using remote actors. That distributed management kind of thing is not still there.

Yes. Besides Akka I've have worked on a couple of other open source projects. Those are mostly mine. One is Skson, Scala JSon implementation, Json serialization, deserialization library in Scala. Then the Scala driver for Couch DB, scouchdb, and that is also on my github account, these are being used by many people right now. Plus I also have an implementation of a Scala redis driver. So these are some of the things which are my side projects.

Yes, actually it used the protocol which Couch DB uses, so it’s just a wrapping over the protocol. One significant value addition is that in Couch DB natively you can write your views using Javascript, but their architecture is such that the view server is decoupled from the main server. So in my implementation you can write the views in Scala also. So you can write your view scripts, map/reduce things in Scala and use Couch DB as the back end.

I'm trying to learn Haskell. I also like to program in Erlang. Clojure is also in fact one of my favorite languages. I like the syntax of Closure a lot, the uniformity in the syntax, the succinctness which the syntax has, I like it a lot.

I don’t combine, but there are some problems which can be better solved using dynamic typing, but if I am to model a complex domain model, I would go for a static typing world. There are some problems, maybe the controller layer or something like that, where I find the dynamically typed languages may be a better fit. So Clojure comes very close there as one of my favorites.