Search / Analytics / Distributed Systems / Machine Learning / DSLs

Summary

In this post we will see how to use docker-compose and sbt-docker to deploy scala microservices. We will create microservices to (1) get tweets using twitter streaming api and put them in kafka and (2) read from kafka and get number of hashtags in each tweet. We will then see how to use sbt-docker plugin to create separate docker images for our services. Finally we will use docker-compose to define the environment for our services and run them.

In order to get optimal performance from cassandra, its important to understand how it stores the data on disk. Its common problem among new users coming from RDBMS to not consider the queries while designing their column families(a.k.a tables). Cassandra’s cql interface return data in tabular format and it might give the illusion that we can query it just like any RDBMS, but that’s not the case.

Domain specific languages, when done right, helps a lot in improving developer productivity. First thing which you need while creating a DSL is a parser which can takes a piece of text and transforms it in structured format(like Abstract Syntax Tree) so that your program can understand and do something useful with it. DSL tends to stay for years so while choosing a tool for creating parser for you DSL you need to make sure that its easy to maintain and evolve the language. For parsing simple DSL, you can just use regular expression or scala’s in-built parser-combinators, but for even slightly complex DSL, both of these becomes performance and mantainenance nightmares.

In previous post, we saw how to create and parse DSL with antlr4. In this post we will compare the two tree walking mechanism provided by the library - Listener vs Visitor. Both approaches have their own advantages, and choice of preferred method depends on what you are using antlr for.