About this blog

About me

Marcin Zajączkowski

Software Craftsman & Solution Architect

I am an experienced architect who specializes in creating high quality software. Being under the impression of the Agile methodologies and the Software Craftsmanship movement, I believe in the value of good, testable and maintainable code. I aim to forge good software that makes the client delighted and the team proud of how the code itself looks.

In my teaching, as a conference speaker, college lecturer, IT coach and trainer, I show how to guide software development effectively using tests (with TDD, pair programming, Clean Code, design patterns, etc.) and maintaining a quality-oriented development environment (with CI, Sonar, automatic deployment, etc.).

I am also a FOSS projects author and contributor, a Linux enthusiast.

Living in Warsaw, Poland.

Email Subscription

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

In my previous post I presented how to display and filter dependencies in multi-module Gradle build. This time I will show how to quickly discover why become a dependency of our project.

Problem

Real life use case. Multi-project Gradle build. In the runtime SLF4J reports problem with two discovered implementations: slf4j-logback and slf4j-simple. Logback is used in the project, but where slf4j-simple came from? Of course it is not listed in our build.gradle, but it is packaged into the WAR file and makes a conflict.

Long and bumpy way

With the knowledge from the previous post one of the possible solutions is to write allDeps task, dump dependencies to file and find a rogue dependency.

It is not pretty visible on the first sight even for that small project with only 4 direct dependencies. But luckily there is a better way.

Quick solution

In addition to dependency task (implemented in DependencyReportTask), Gradle has one more similar task – dependencyInsight (implemented in DependencyInsightReportTask. It allows to limit a dependencies tree only to selected dependency (also transitive).

Summary

dependencyInsight task can be very useful when tracking down suspicious and not expected transitive dependencies in the project. An ability to make it multi-project build friendly makes it even more powerful.

gradle dependencies allows to display dependencies in your project printed as pretty ascii tree. Unfortunately it does not work well for submodules in multi-project build. I was not able to find satisfactory solution on the web, so after worked out my own that blog post arose.

Multiple subprojects

For multi-project builds gradle dependencies called in the root directory unexpectedly displays no dependencies:

No dependencies displayed for the root project

In fact Gradle is right. Root project usually has no code and no compile or runtime dependencies. Only in case of using plugins there could be some additional configurations created by them.

You could think about --recursive or --with-submodules flags, but they do not exist. It is possible to display dependencies for subprojects with “gradle sub1:dependencies” and “gradle sub2:dependencies“, but this is very manual and unpractical for more than a few modules. We could write a shell script, but having regard to (potential) recursive folders traversal there are some catches. Gradle claims to be very extensible with its Groovy based DSL, so why not take advantage of that. Iteration over subprojects can give some effects, but after testing a few conception I ended with pure and simple:

subprojects {
task allDeps(type: DependencyReportTask) {}
}

When called gradle allDeps it executes dependencies task on all subprojects.

Dependencies for all subprojects

Remove duplication

All dependencies belong to us, but some parts of the tree looks similar (and duplication is a bad thing). Especially configurations default, compile and runtime and the second group testCompile and testRuntime in most cases contain (almost) the same set of dependencies. To make the output shorter we could limit it to runtime (or in case of test dependencies testRuntime). dependencies task provides convenient parameter --configuration and to focus on test dependencies “gradle allDeps --configuration testRuntime” can be used.

Dependencies in one configuration for all subprojects

Summary

Where it could be useful? Recently I was pair programming with my old-new colleague in a new project (with dozens submodules) where SLF4J in addition to expected slf4j-logback provider discovered on a classpath also slf4j-simple. We wanted to figure out which library depends on it. Logging dependencies tree to file with a help of grep gave us the answer.

As a bonus during my fights with DependencyReportTask I found an easier way how get know who requires given library. I will write about it in my next post.

When writing integration tests it is sometimes required to set up environment initial conditions/state before/after a given test or the whole specification. Upcoming Spock 1.0 expands the number of available options to do it in the convenient way.

This is the second part of the series about new and noteworthy in (upcoming) Spock 1.0.

New extension @RestoreSystemProperties

System properties provides information about JVM configuration and the environment like JVM vendor and version, operating system, classpath, locale or a time zone. Some of them can impact the way our application works. The following example checks if the protection before running a dangerous program as root works properly on Unix machines. To not affect other tests @RestoreSystemProperties restores the original system properties.

The extension can be activated using @RestoreSystemProperties annotation. It can be placed on a feature method to be applied right after the given test only or on a class to restore system properties after every test in the specification. The behavior in the second case is identical to placing the annotation on every test method.

Internally Spock makes a copy of a system Properties structure before a test and restores it after.

@ConfineMetaClassChanges (since 0.7)

One of the things which make Groovy language so powerful is metaprogramming – an ability to modify classes (e.g. add/modify new methods) at runtime. That could look like hacking (and that is true), but in some specific cases it is very useful (see Grails and GORM for creating database queries when non existing methods – like findByFirstNameAndSecondName(...) or writing a custom DSL like 6.minutes.ago).

MetaClass operations are usually executed at the class (not instance) level what generally causes that they affect every class instance in given class loader. GORM like changes done in multiple test could interact each other causing tests to fail. For that cases @ConfineMetaClassChanges extension (available already in previous Spock version) was created.

@ConfineMetaClassChanges annotation can be placed on a feature method to restore MetaClass to the state just before that test or on a class to restore system properties after the last test in the specification to the state before calling the setupSpec method.

Note that @ConfineMetaClassChanges behavior placed on a specification level is different (once after the last test) than @RestoreSystemProperties behavior (every time after every test).

@AutoCleanup (since 0.7)

Another already existing, but rarely known and used extension is @AutoCleanup. Usually external resources are allocated in setup/setupSpec methods and released in cleanup/cleanupSpec methods. But Spock, the same as pure JUnit, support auto initialization instance variables (fields) before every test/feature (which overrides default Java behavior when a field is initialized only once when a class instance is created).

@AutoCleanup supports both instance variables (the method is called after every tests) and static/@Shared fields (the clean up is performed after the last test). All errors during clean up are caught to not interrupt test execution. By default there are logged, which can be disabled using quiet parameter @AutoCleanup(quiet = true).

Where can I find Spock 1.0?

Please notice that blog post is about features available in Git branch planned to become Spock 1.0 sometime in the future. They are currently available only in a 1.0-SNAPSHOT version in a separate repository. As not being released (at the time of writing) they could look different or even (in the extreme case) be not available in the final version. Be aware.

Summary

Mentioned extensions were designed for integration tests and because of class loader (or system) wide changes it is not safe to execute more than one test in the same time. Therefore it is important to separate pure, very fast and independent, unit tests from integration tests to (inter alia) allow unit tests to run in parallel.

After @Require and @IgnoreIf this is the second post about changes in upcoming Spock 1.0. The examples was written using Spock 1.0-groovy-2.0-SNAPSHOT and they can cloned from GitHub.

Introduction

Some tests (especially integration tests) should be run only if certain conditions are (or are not) met. Upcoming Spock 1.0 provides new @Requires and improved @IgnoreIf extensions to handle that requirement in a convenient way.

Historical background – waiting for Spock 1.0

The latest stable Spock version (0.7) has been released in October 2012 (~2 years as the time of writing). There have been made hundreds of commits to Git repository since then and many new features and improvements are already available in the current SNAPSHOT version. Unfortunately the “New and Noteworthy” section in the documentation doesn’t cover those changes and the only way (as I know) to get them know is digging into Git commits. As a Spock user I was curious what new is around the corner, so I did that job which resulted in the presentation at Confitura (in Polish). A as bonus I planned a series of blog entries describing those more interesting features and possibly also an update of Spock documentation.

This is just the first part of the changes in (upcoming) Spock 1.0. There is much more to discover and I will be presenting that in future blog posts.

New extension @Requires

@Requires allows to run given test (or the whole specification) only if given criteria are met.

It is opposite to already known from Spock 0.7 @Ignore extension which allowed to ignore the given test (or the whole specification) using access to system properties (properties – System.getProperties()), environment variables (env – System.getenv()) and java version (javaVersion – System.getProperty("java.version")). In Spock 1.0 branch they both are much more powerful.

Testing my examples in practice I was surprised that a system property “os.arch” returns not a processor architecture taken from an OS, but a JVM version. So, be aware that running a 32-bit JVM on 64-bit system will return “i386″. Hopefully usually that knowledge is not important (unless for example the native libraries are used).

New features in @Requires and @IgnoreIf

In addition to the new @Requires extension it and its twin @IgnoreIf have got new internal abstraction to simplify the way how the operating system information and a JVM version can be accessed.

Operating system

Operating system information can be accessible in Spock 0.7 using os.name and os.version system properties. Unfortunately it can be complicated and error prone in some cases:

Using static methods

@Requires (the same as @IgnoreIf) can also use static methods to define a condition (which is available, but less known in Spock 0.7). Those methods can be declared inside a given class, in an another class or (when using Groovy 2.3+) in a trait.

Please pay attention to the static import in this example. Even though that code without an import looks ok and Idea sees the method it will fail at runtime with groovy.lang.MissingMethodException: No signature of method: (...). This is caused by the way how Spock internally tries to resolve references in a Closure. For detailed explanation see the next paragraph.

How does it internally work (for the curious)?

You may wonder how that code even compile if jvm or os are not known for the Specification class. The key thing is that code inside @Requires (and @IgnoreIf) annotation is wrapped with {} (curly brackets). In Groovy that syntax means “anonymous code block” and it is named Closure. The @Requires annotation accepts a Closure as a parameter:

Two important aspects of a Groovy Closure are used in Spock to implement those features:

a block of code is executed “at the later point”

an execution context can be delegated

The first point causes that Groovy compiler ignores unresolved references. At the compilation time the execution context can be unknown (a Closure can be passed as a parameter far away from the declaration place). The used references will be resolved at runtime and here the second point applies.

By default property references and methods are attempted to be resolved to the owner (an enclosing class or a surrounding Closure) and later to the delegate (by default the same as owner). A delegate can be changed and that is used in RequiresExtension:

PreconditionContext is a delegate class for @Requires and @IgnoreIf providing an execution context. Its getOS() or getJvm() methods are resolved and executed when os or jvm properties are used in the annotation Closure. In addition to set a dedicated delegate, a resolve strategy is changed to not bother with an enclosing class and try resolve unresolved references only using PreconditionContext.

If you are new to Groovy and feel confused with the internals I propose you to take a look at the introduction to Groovy Closure. If you like it there is a lot more things to get know about Groovy Closure and its power.

Where can I find Spock 1.0?

Please notice that blog post is about features available in Git branch planned to become Spock 1.0 sometime in the future. They are currently available only in a 1.0-SNAPSHOT version in a separate repository. As not being released (as the time of writing) they could look different or even (in the extreme case) be not available in the final version. Be aware.

Summary

New @Requires extension and the enhancements in @IgnoreIf are the first part of the series about new and noteworthy in upcoming Spock 1.0. When will it be released? Citing John Carmack “It’ll be done when it’s done”. If you would like to bring the release closer donate your time and contribute to the project.

Post written using Spock 1.0-groovy-2.0-SNAPSHOT. Examples can be cloned from GitHub.

Simple real life problem. The proper service implementation is taken from a map based on a given service key. For an unknown/unsupported key a meaningful exception should be thrown. How could it be implemented without an if statement?

Business context. PriceProvider allows to get price for the requested product. A concrete implementation call proper provider via Web Service (REST or SOAP). There is a delegate/dispatcher which depending on a given provider name/id selects the proper implementation.

Implementation note. Mapping selected provider to the corresponding provider adapter implementation can be implemented in many ways. An external properties file, a records in a database or even have a separate web interface. However in many cases those connections are rather permanent and do not need to be changed at runtime. Then a simple map is a very compact and efficient solution. No, 6 “if..else..if..” statements are not the option :).

There is a map with pairs: provider id -> provider service. It could be initialized (in Java) for example with:

The problem is that we need to implement all methods from the interface and what is worst we don’t know which provider id caused the situation to put it in the exception.

Then I recalled this code is in Groovy and my thoughts went to closure coercion. At the end it turned out that Groovy has a very nice mechanism for it. We can use Map.withDefault(Closure) method which allows to define logic to calculate returned value for unknown keys.

Technical insight. Under the hood Groovy creates a wrapper which for unknown keys calls the closure and put its execution result into a map (for given key). Therefor even for complicated logic its cost is paid only once – on the next query the value is got directly from a map.

Unfortunately my formatting style was not supported in IntelliJ Idea, even with idea-spock-enhancements plugin. This forced people like me to format it manually and pay attention to automatic code reformat.

Idea 13 provides a lot of new features and improvements. One of them is:

Btw, one more thing pointed out by @mariusz_s. By default labels in Idea looks like ordinal text. In Spock tests labels are part of the specification language and it would be nice to stand them out. It is possible with Spock Enhancements plugin for Idea which make the labels bold and colored.

The plugin provides also live inspections for block ordering errors which is very handy and I suggest it to all writing tests in Spock. Code long and prosper ;)

AssertJ and Awaitility are two of my favorites tools using in automatic code testing. Unfortunately until recently it was not possible to use it together. But then Java 8 entered the game and several dozens lines of code was enough to make it happen in Awaility 1.6.0.

AssertJ provides a rich set of assertions with very helpful error messages, all available though the fluent type aware API. Awaitility allows to express expectations of asynchronous calls in a concise and easy to read way leveraging an active wait pattern which shortens the duration of tests (no more sleep(5000)!).

The idea to use it together came into my mind a year ago when I was working on an algo trading project using Complex event processing (CEP) and I didn’t like to learn Hamcrest assertion just for asynchronous tests with Awaitility. I was able to do a working PoC, but it required to make some significant duplication in AssertJ (then FEST Assert) code and I shelved the idea. A month ago I was preparing my presentation about asynchronous code testing for the 4Developers conference and asked myself a question: How Java 8 could simplify the usage of Awaitility?

For the few examples I will use asynchronousMessageQueue which can be used to send ping request and return number of received packets. One of the ways to test it with Awaitility in Java 7 (besides proxy based condition) is to create a Callable class instance:

and thanks to the new AssertionCondition initially hacked within a few minutes it became a reality in Awaitility 1.6.0. Of course AssertJ fluent API and meaningful failure messages for different data types are preserved.

As a bonus all assertions that throw AssertionError (so particularly TestNG and JUnit standard assertions) can be used in the lambda expression as well (but I don’t know anyone who came back to “standard” assertion knowing the power of AssertJ).

The nice thing is that the changes itself leverage Runnable class to implement lambdas and AssertJ support and Awaitility 1.6.0 is still Java 5 compatible. Nevertheless for the sake of readability it is only sensible to use the new constructions in Java 8 based projects.

Mutation testing allows to check the quality (effectiveness) of automatic tests. PIT (aka pitest) is a leading mutation testing tool for Java environment. In my last blog post about PIT in January 2013 I have covered version 0.29. Since then the PIT development team has been busy and the 4 releases introduced various new features (besides fixed bugs). In this post I will cover the most important (in my opinion) changes in the project (up to recently released version 0.33).

New features

– preliminary support for Java 8 bytecode – PIT can be used with code which contains Java 8 syntax and constructions (including lamdas)
– internal refactoring resulted in much faster “standard” line coverage calculation
– support for parametrized JUnit tests written with Spock (in Groovy) and JUnitParams
– ability to define a coverage threshold (both line and mutation) below which the build will fail
– ability to use PIT with Robolectric
– new Remove Conditionals Mutator (a conditional statement will always be true – not enabled by default as of 0.33)
– new Remove Increments Mutator (an increment operation will be removed – not enabled by default as of 0.33)
– ability to choose JVM to be used for mutation testing
– ability to run PIT only for locally changed files for Maven build with configured SCM plugin
– demanding users can define their own strategies for: test selection, output format and test prioritization – PIT provides extension points which allow to write custom implementations
– partial support for JUnit categories
– support for mutating static initializers in TestNG

In the meantime there were also releases of plugins/tools based on PIT. My plugin for Gradle was enhanced with the dynamic task dependencies resolution (just “gradle pitest” takes care about all the requisites in the Gradle build lifecycle) and support for the additional main and test source sets. Plugin for Eclipse has got (inter alia) a new mutation view and an ability to run PIT against all the projects in a workspace.

Not only releases

Besides new releases PIT has got brand new Bootstrap based webpage, the logo (see above) and the source code was migrated from Mercurial on Google Code to GitHub. The nice thing is that the move resulted in a few contributions withing the first weeks.

Henry Coles the author of PIT also started the new commercial project FaultSeed – “better mutation testing tools for the JVM” which will be based on PIT and has a goal to be 50% faster than PIT and support also Groovy and Scala. Very promising.

PIT (and mutation testing in general) becomes more and more popular and recently there were given a number of talks about it (including my talk at Developer Conference 2014 – slides). The number of questions on the project’s mailing list also significantly increased. And you, have you tried PIT in your project yet?

Mockito uses a lazy approach for stubbing and when a not stubbed method is called it returns a default value instead of throwing an exception (like EasyMock). This is very useful to not overspecify the test.

A default returned value depends on a return type of a stubbed method. For methods eturning collections we have an empty collection, for numbers – 0, for booleans false, for ordinary objects – null (in Mockito 2.0 the set of not null values will be extended – this can be also achieved with 1.9.x and ReturnsMoreEmptyValues answer).

Before we go any further a quick introduction do Answers (you can skip to the next paragraph if Answers are for you like an open book). In addition to simple stubbing based on desired value passed to Mockito directly:

the stubbing API provides a way to pass an object with the logic to determine what should be returned in given case (based on method arguments or even an internal state (for consecutive calls)). A simple practical example returning always the first parameter passed to the called method:

Mockito provides a set of build-in answers. Some of them (like ThrowsException or CallRealMethods) are used by Mockito internally, but some others (like ReturnsArgumentAt introduced in 1.9.5) can be also useful for developers writing tests.

Let’s return to the main topic. Sometimes it is useful to change those default values. In addition to using the answer mechanism for stubbing a specific method calls Mockito provides a way to specify an answer which will be used for every not stubbed method execution on given mock. To do so we can use a static mock() method which in addition to a class to mock takes an additional parameter – a default answer.

mock(SpaceShip.class, Mockito.RETURNS_DEFAULTS);

As returns defaults is a default behavior in Mockito above code is just a more explicit version of:

mock(SpaceShip.class);

but we can use this construction to achieve a few interesting behaviors. One of the predefined answers provided by Mockito is RETURNS_DEEP_STUBS. It causes an automatic stubbing of chained methods calls and allows to do following:

Please note that with default configuration it would cause NullPointerException due to the fact spaceShipMock.getTacticalStation() returns null. With RETURNS_DEEP_STUBS Mockito under the hood creates a mock for every middle method call. This is an equivalent of:

As a bonus, deep stubbing allows to perform a verification (only) on the last mock in the chain:

verify(spaceShipMock.getTacticalStation()).getNumberOfTubes();

Another provided answer is RETURNS_MOCKS. This tries to return default value using ReturnsMoreEmptyValues answer (an extended version of a default ReturnsEmptyValues), but if it fails a mock is returned. Only in the situation where the return type cannot be mocked (e.g. is final) null is returned.

mock(OperationsStation.class, Mockito.RETURNS_MOCKS);

Sometimes it can be useful to stub specified methods, but delegate remaining calls to the real implementations. It can be done with CALLS_REAL_METHODS. It can be useful for example when testing an abstract class (just the implemented methods without the need to subclass to create a concrete subclass).

mock(AbstractClass.class, Mockito.CALLS_REAL_METHODS);

Please note that using RETURN_DEEP_STUBS, RETURN_MOCKS and CALLS_REAL_METHODS should be not needed when dealing with well crafted code, written with the usage of Test-Driven Development. Nevertheless sometimes it is required to write tests for legacy code before a try to refactor it.

From a set of default answers defined in Mockito.java, there is also a very useful RETURNS_SMART_NULLS option. This returns SmartNull class instance instead of plain null, which provides a hint which mock stubbing was not performed correctly (and caused NPE). I wrote more about this mode some time ago in Beyond the Mockito Refcard #1.

In addition to define a default answer we can use any class which implements org.mockito.stubbing.Answer interface – both provided by Mockito or hand written. One more tip. In case you would like to use RETURNS_SMART_NULLS or ReturnsMoreEmptyValues globally for all mocks in your application you can check a trick with MockitoConfiguration.

Btw, in case you are starting an adventure with Mockito or want to learn more or just want to organize your knowledge you can be interested in my Mockito Refcard available for free from dzone.com.

Btw2, in addition if you are new to Mockito and live near Warszawa you can consider an attendance in my lecture/workshop about Mockito at Jinkubator – 18 II 2014 (next Tuesday).

AppFuse is a full-stack framework for building web applications on the JVM. It was originally developed to eliminate the ramp-up time found when building new web applications for customers. Over the years, it has matured into a very testable and secure system for creating Java-based webapps. At its core, AppFuse is a project skeleton, similar to the one that’s created by your IDE when you click through a wizard to create a new web project.

I you are looking for some foundation for your project or just would like to see how the same things look in different technologies you can give AppFuse a try. A quick start page should be a good starting point.

There is also a personal thread in this story. Steadfast readers can remember that over 3 years ago I started working on Wicket frontend for AppFuse. I definitely prefer working on backends, but I wanted to get know Wicket (and its famous ability to being tested without Selenium) better and an engagement in the AppFuse development seemed to be a good way to practice these skills. There are still places in a Wicket frontend which need to be polished, but the work is mostly done (what I’m happy about :) ).

As a summary of my work I can write that even Wicket (where all page logic is written in Java or Groovy classes – no more c:forEach tags!) cannot completely remove the pain which comes with the limitation of HTTP (which wasn’t designed for the “enterprise applications”) and differences between browsers (although with jQuery and Bootstrap it is much easier). In addition 3 years is a lot of time in IT and currently there are even more use cases where component-based server side frameworks aren’t the best solution to make a good looking, responsible, scalable and trendy UI. Maybe it is time to work on an AngularJS frontend ;).