Feb 8, 2015 •

If you work in an organization or doing your personal project sooner or latter
you’re gonna need an artifact server. And few reasons this would be you want to
host your build artifacts but not make those available to the public, you want
to have a proxy server to use when you download your artifacts, this is
particularly
helpful when you have more than one person working on the project or you simply
have some jars that aren’t in maven central or other public repos and you want
to host those so you can list them as dependencies from your build tool.

Feb 28, 2015 •

So what is continuous integration, at the very high level it’s about hey is
the code in a good state, no matter by how many people code is being updated,
how often code is being changed or of how many building blocks project consists of.
When you start working on a feature you’re gonna branch of mainline and start
introducing changes. At some point when you find things are broken now you have
to spend time to see if it’s because of your changes or project was already in
a bad state. And when you find out that it was already in a bad state you
would want to start tracing. A who introduced the breaking changes and B when.
So to avoid the head each and frustration what you really want to know is
if project is in a good state before you even start working on it.
One option would be to compile the code before you start changing it.
But if code compiles is it enough? Oh you you also wanna know if unit tests are
passing. But is this enough? Check for compiler warnings? Style violations?
Code coverage? What if you run code coverage and it shows that it’s 85%,
does it mean everything is good, what if code coverage was 95% last week and
code without proper code coverage got committed and code coverage dropped
(do you remember what code coverage was last week, can you even find
out without going back in commits and running coverage reports on those commits.) are you sure
you want to introduce your changes on top of those changes?
Let’s make it even more complex, what if the project consists of many other
projects and the project you’re working on brings all those dependencies
tougher. Maybe you should checkout all those dependency projects
and see if those are in a good state also.
Let’s make it even more complex, what if the project isn’t just set of
projects that get bundled into one big project.
What if the whole code base consists of many micro services that talk to each other
via some form of PRC. OK maybe you can manage to do all this to know if code is
in a good state before you introduce your changes what about the point B I
brought earlier about when code broke, because longer it was in that states
longer it’s gonna take to trace the breaking change and longer to fix.
Can you go and check every single project every time someone makes a commit
to make sure that commit is good? Well actually you can, you just need a
build server. So all those points I was bringing was about whatever your
integration is it should be continuous and it’s continuous if it runs after
somebody changes something (or maybe before is a better option?☺ I’ll get
to this later.)

May 22, 2015 •

Recently I needed to cache some lightweight data into actor’s local memory to
not make unnecessary calls to third party service over the internet. I think what I
came up with is a pretty simple implementation (hence I named the article
simple caching in akka). But the idea can be expended with small changes
if any to use it to cache actor messages. Let’s imagine the word of actors where
we have one actor per user and we send messages to user actors to get some info
back. Now when we say each user (actor) is responsible to cache it’s own data
it means each actor will use even more memory so at some point we’ll need to
scale out because we’ll run out of memory (not that we didn’t have that
problem without cache at first place). But it’s a good thing that we know we
need to scale out from the beginning because I didn’t mention about high
availability yet, so we at least need two boxes.

May 31, 2015 •

Over time different libraries brought different testing styles, jUnit, xUnit,
Rspec, ert. Nowadays since many people have different backgrounds and
different taste when it comes to writing unit tests, most test libraries
offer many testing styles and ScalaTest isn’t an exception. To me there are
three groups of testing styles. One would the traditional jUnit, xUnit
style where you define a test class and unit tests are methods in that test
class, you also define setUp and tearDown methods that must be called
before and after each unit test. Of course there are also setUp and tearDown
versions that run once per test suit but I don’t want to talk more about it here.
I must admit that this isn’t my favorite style and here is why. I like to unit
test aggressively without skipping any business cases instead of just
relying on code coverage and saying my other test already is hitting those
lines. If I have a class with many methods I write many unit tests for each
method, an example would be when a method takes a parameter and based on
different values for those parameters it does different things, of course we
want to have multiple unit tests for that method to cover all business cases
for that method. So this alone groups unit tests in a test class by functions,
another thing is that most likely unit tests for function A will share some
setup logic. This testing style forces as to put setup code for all unit tests for function A in
setup method, and when we have like 10 such functions, setup method will have
the setup code for all those function which will make it long, hard to read,
also it will be extremely hard to tell which setup code is for which unit test.
Another option is to define setup code in each unit test with isn’t really a
good idea since one we’ll have duplicated code and unit tests now will be
longer than they need to be and hard to understand. I see some people like to
solve this problem by moving this common code into private methods. Although this
solves code duplication problem, also makes unit tests shorter but it introduced
another big problem. When I read a unit test I want to know what the method is
doing in 2 seconds, unit tests are documentations right. Now you come across
this private method call, you have to find it’s implementation, read the ugly
code there and jump back to continue reading what that unit test is trying to
test. It’s a nightmare to me, I call that unit test anything but readable and
I sure don’t want to be one maintain that code.
Before I moved to next type of unit tests, I must mention what styles ScalaTest
is offering here. FunSuite and FlatSpec, you can read more about those in
ScalaTest documentation. Both are flat as mentioned, one is more like jUnit,
other is more like xUnit.

Jun 16, 2015 •

When activator just came out I wasn’t really excited about it. I’m more of a
command line guy and using a web UI to help me build, test, run my code isn’t
very useful to me. Activate also allows you to browse the code from web IDE. I
usually use Vim, IntelliJ or GitHub for it, but I get the idea, for someone who
doesn’t have his or her environment setup and maybe got a project that’s not
even in github yet, web IDE can be helpful. I have to give more credit to
activator though, because of couple really cool things. Recently my colleague
who’s not a scala developer (and of course he doesn’t have his scala
environment setup with latest sbt and all that) needed to run one of the
projects we have in his local computer and asked me how to do it.
And of course first thing that came to my mind was get the latest sbt and do sbt
run. Then I remembered that I started that project from one of the activator
templates I created, so it must have activator there. So he ran ./activator
and magic! It went and downloaded needed sbt version and loaded sbt environment
for him. And the reason I started my own activator templates at first place is
because it solves another big problem for me.

Jun 24, 2015 •

With akka becoming more and more popular, akka kernel being deprecated,
docker rocking the word, Typesafe releasing ConductR and more folks
getting to scala, I wanted to write a post about how to run akka applications.
Akka applications aren’t servlet applications java developers are used to,
normally those are main class applications in a jar file and have other jars as
dependencies. So it’s no-brainer that we can just use java command and
no container like tomcat or jetty is need. I will cover how to run in
development while iterating on a task. How to run it in development and tunnel
the port and let others point to your code before you even go to production
or pre production environment and how to run it inside a docker container in
production and pre production environments. Unforunatly I didn’t get hold of
ConductR to try it myself, so I won’t be writing about it in
this post. Btw, most of this should be to relevant other scala/java main class
applications not only akka and akka http applications.

Jul 6, 2015 •

Scalaz has so many abstractions and higher kinds that model lot of
algebraic structures, monads, hight level types to generalize common structures
and offers lot of tools that work with those abstractions.
In this article I wanted to cover the very basics. Ones we see every day
in standard scala library. What the anomalies of basic types are in standard
scala library, what are the comment things between them that can be made
generic, how to use scalaz with standard classes defined in scala library.

Jul 6, 2015 •

Scala is an object oriented language with functional aspects. Some people
like the object oriented aspect, some are attracted by the functional part.
Add to that the fact that it’s sitting on top of JVM and of course it’s
influenced by Java a lot, we still have nulls, we still have exceptions,
IO and state mutation, etc. And since Scala isn’t purely functional it still
allows us to write imperative code, call a Java library that will throw an
exception, etc. I personally think that functional code is easier to read and
understand, unit test and maintain(not event talking about state and multiple
threads). So when it comes to writing purely functional code (or as close to it
as possible) Scala sure misses certain things and scalaz has a lot built to
help. I decided to write series or posts to explain scalaz. I’ll start with the
basics, most common abstractions then explain more real words cases.
I’ll put the links to all posts here so it’s easy to navigate.

Jul 13, 2015 •

In my previous article about Semigroup, Monoid, Equal, Order, Enum
and Show. I covered what those are, what method they have and what derived
and syntax sugar methods scalaz bring for them. Also I showed that there are
implicit implementations of those for scala standard classes that once are
brought into the scope you’ll have access to those. In this article I’m gonna
show those methods in action for integer, string and boolean.

Sep 16, 2015 •

Here at Ad Hoc Labs we are open-sourcing one of the
projects I started to handle reliable execution of different tasks.

I wanted to build a library that would:

have a higher level of abstraction than queuing

allow development of new types of tasks with only config changes

be back-end independent, so backends like RabbitMQ and Kafka would work based on config

allow flexibility to have publishers running in one project and consumers running in another

allow flexibility to decide which projects run what consumers and producers and how many of them.

Our product Burner is a privacy layer for phone numbers.
There are many things we want to run in a reliable manner, and we need to queue tasks and
execute things in an asynchronous way. We are a Scala shop and among many Scala libraries
we also use Akka.

In the Akka world, we have actor A sending a message to actor B. There are many things that
can go wrong, even if both are running in same JVM. Actor B may be down when actor A is
sending the message. Actor B may fail to process the task, so there is a need for retries.
Some tasks can take long time to process, so there is a need to run things asynchronously.
You may want to scale out and have many instances of actor B, etc.

This actor model is very clean and well-suited for this type of situation. And if
you’re familiar with Akka, you know that Akka handles some of those cases out of the box,
some via Akka persistence, and some things simply aren’t there.

So I wanted to have a very simple library that wouldn’t require a Cassandra cluster, force
you to write a lot of boilerplate, or force the use one specific message queue – I wanted it
to be extremely flexible and extendable.

The core idea is the same: actor A sends a task to actor B to process, and actor B
will eventually get the task. It’s very similar to the way you send a message from actor A
to actor B in Akka; there’s no other complexity there.

Sep 24, 2015 •

Applicative is Apply that also has point or pure methods.
Scalaz defines Applicative[F[_]] trait with point abstract method.

defpoint[A](a:=>A):F[A]

point method lifts A to F[A].
Since Applicative is Apply it inherits all the methods that Apply
offers plus it needs to provide implementation for map and ap methods.
Note that Applicative implements map using point and ap methods:

Note that in Applicative, map method is implemented using
ap method and in Bind, ap is implemented using bind method. With that,
two methods that are left to be implemented for a Monad are
point and bind.

Jun 4, 2016 •

Jun 5, 2016 •

Ammonite is such a great project, especially Ammonite-REPL. It brings many great things to scala console: syntax highlighting, better editing, dynamically loading dependencies, loading scripts, etc. It comes very handy if you want to play with a some library without creating an sbt project. Or you want to try something out without necessarily loading IntelliJ. I happen to do it often and decided to start an open source project called ammonite-modules. The idea is to have pre-defined modules you can load in Ammonite Repl. Modules will have all the dependencies needed at latest versions and do some basic setup.

Jul 13, 2016 •

Akka Http is a great framework for rest microservices and here at Ad Hoc Labs we’re slowly migrating from Spray to Akka Http. Since we use Spray extensively we ended up having utility classes to complement some functionality here and there or shared in a common jar with other utility classes. With Akka Http we want to keep things little more clean and reduce boilerplate with high level constructs. We decided to start a new project named akka-http-contrib that will host all akka http functionality that we’re building that’s not already in akka-http project. We recently open sourced it since other community members will find it helpful and hopefully will contribute also.

First functionality we needed for a new project we started with Akka Http was to introduce throttling for some endpoints. We didn’t want to write boilerplate for each endpoint we want to introduce throttling for but instead we wanted to be able to introduce throttling for any endpoint by adding it to typesafe config. Basically we wanted a directive that will wrap the routes and when request comes in, depending on the config and thresholds, either let the route execute or complete the request with 429 - Too Many Requests and prevent the route from being executed. This at high level, which should handle most of the use cases but we also wanted to have it extensible enough that whenever there is a special usecase it still can be handled by writing little code.

Oct 16, 2016 •

AWS Lambda supports writing handlers in Nodejs, Python and Java. The first two are really easy - if you don’t have any external dependencies and script is easy to write, you can even write your lambda in AWS console. However when it comes to Java, things get a little more complicated than that (not even talking about Scala yet.) So at the very least you’ll need to create a new project, know what jar dependencies to add for lambda, implement the handler method in a class that potentially implements some interface, know how to generate a deployment package (fat jar or a zip file for java projects), upload to S3 (since that jar would have a decent size), etc. If I want to implement my lambda in Scala now, I would have to deal with all those complexities for Java projects and more - such as more dependencies, how to write an actual Scala code (not Java in Scala syntax), interoperability, immutability, built in features like case classes, futures, etc. I thought it’s not fair for Scala developers and things can be much simpler (maybe not Nodejs simple) but at least it has to be simpler than doing in Java, since we all know Scala has more things to make developer’s job easier.

Oct 29, 2017 •

I had few activater templates for a while now that I use to start different types of Scala projects. I open sourced those and made available via Github and activator website. There was lots of traction from the community and since Activator is deprecated now I decided to take those and convert to Gitter8 templates so sbt new can be used to start projects. The new templates work with sbt 1 and all the dependencies, including sbt plugins in the projects, are upgraded.