Primary Menu

All posts by chris

The Maritime Dev Con was a huge success. About 95 people total attended the event making it a huge success for developers in the Maritimes. I had a great time at the event and met a bunch of really cool people.

The presentations I gave went well with a number of attendees. I’m putting up the slides from the presentation here in case you want to review them.

I’m also going to include the sample code from the presentations. The samples was for the modern hello world example in Java and Groovy. It used the twitter API to query MaritimeDevCon from twitter to find my ‘Hello World’ tweet.

There is going to be a maritime developers conference coming up on June 18th in Moncton. It is going to be a great opportunity to have developers from Moncton and other areas of the maritimes get together and learn a bit about other languages and technologies they might not have been exposed to. All of the presentations are limited to 45 min and will mostly give an introduction to the language or technology.

Modern Java Development – In this presentation I’m going to give an introduction to Java for non-Java developers. It will cover the basics of knowing where to start and getting started.

Groovy Primer – This is essentially going to be a rehash of the Groovy talk I gave at the Maritime Java User’s Group a month or so ago. This will focus on showing what Groovy has to offer (particularly to Java developers) and how to get started with Groovy.

Last night I gave a talk at the Maritimes Java User Group in Moncton. The presentation was an introduction to Groovy for Java Developers. I had initially done a 3h internal course for iWave Software to bring them up to speed on what Groovy is and why we should use it. I took this course and cut the size down a bit (about 1h 15min + questions) for the presentation last night.

It was a great third event for the Maritimes JUG in Moncton. We ended up with a few less people than we hoped but everything learned something and had a good time. In case you missed the talk or want a review, I have made the slides used for this presentation available.

Problem. Bugs happen. The common solution to this problem is to fix the bug and release a patch. Version 1.0 has bugs, version 1.0.1 fixes those bugs.

Inevitably at some point in time you will need to put together a list of all of the changes in a release. For me, this needs to go into a format we can post on our wiki. This process can be tedious if it is a manual process. There are a few approaches to handling this. You can go against the bug tracking repository and look for what bugs were fixed for this release. This will tell you everything that should have changed. I say ‘should have’ here because you cannot know for sure if the information is 100% accurate.

The other option is to go to the version control repository for information on what has change. This is the authoritative source of what has changed but often contains more information than what you would want in a change-log.

In my previous post on version control I mentioned that we have best practices around format for commit messages. All bugs start with the words “Bug

The ‘–xml’ option is used to format the output as XML. This allows groovy to break it down easily.

The ‘-g’ option is used which shows log messages from other revisions that were merged onto this branch. Let’s say you have 50 bugs that are merged onto the bug fix branch all at once. This would create a single revision on the branch. Using this option includes all 50 comments from their original commit on the trunk. This detail we want in the change-log. This gives nested entries though so the code has to handle that case.

The ‘-r’ option is used to specify the revision range to use. In this case for a branch, we want from the previous release revision number to the current (or HEAD). For this example, let’s assume the 1.0 branch was at revision 1528.

The command to run then becomes:

svn log -r HEAD:1528 -g --xml

The next step that needs to be done is to take this XML and turn it into a change-log. I plan to use this as a comment into a wiki so I prefix the lines with ‘*’ so they will appear as a bulleted list in trac. It also puts the revision number at the end of the line in brackets. The output should look like this:

To generate this changelog, I wrote a groovy script. It uses the svn command to generate the changelog and uses Groovy’s XML Parsing to break it up and format it. The path to the working directory and revision number would change from release to release but the rest of the code is reusable.

Two of the most useful tools to a developer outside of their development environment are version control and bug tracking systems. Version control allows tracking of changes to the product and allows for branching and merging. Bug tracking systems allow for tracking issues with the product whether they be bugs or enhancements.

Even though these tools are often separate products, they have a major commonality which is the code you are working with. Often times you want to be able to see for any given bug number, what code was changed for that bug. Also, for a change in the code (in version control) you want to see if it was associated with a particular issue in the bug tracking software.

At the company I work for we use Subversion for version control and Bugzilla for bug tracking. We have some best practices around these tools to make things easier.

Version Control and Bug Tracking Best Practices

When resolving issues in the bug tracking database, our team always puts in the build number of the build that contains the fix. This way a person who is looking at the bug can know if the build they have contains the fix. Anytime our team fixes a bug we put in a comment that looks like this:

Build Fixed: 1.0.1.12354

The last number is the revision number in Subversion.

When we commit code changes to Subversion, we also include the bug number for the bug being fixed. Our commit messages always appear in this format:

Bug 1234: Fixed this bug

Subversion Tooling

Recently I came across a neat feature in Subversion that allows you to link it to a bug tracking system. Basically this allows clicking on the bug number in the subversion history view to take you directly to the bug number in the bug tracking software.

Enabling this feature is fairly simple to do and involves setting 2 properties in the subversion repository. These properties need to be set on the root folder in subversion that you would use to checkout your project from. It automatically is available for everything in that tree but you need to checkout from this root for it to work. These are the two properties that need to be set.

bugtraq:logregex – This defines a regular expression to ‘match’ bug numbers in subversion comments. For the pattern I listed above, we are using: [Bb][Uu][Gg] (\d+)

bugtraq:url – This defines a URL to go to when the user clicks on a bug number. The browser is launched when the number is clicked on and takes you to this URL replacing the BUGID parameter. For our bugzilla repository we are using: https://some.server.somewhere.localhost/show_bug.cgi?id=%BUGID%

The following steps walk through this process of how to set this up using Tortoise SVN:

On the root folder of your subversion working copy, right click on the folder and click TortoiseSVN -> Properties.

A lot of the work I do deals with integration with a third party system. Each of these systems typically has their own API and mechanisms for handling errors. Often times writing some code to ‘wrap’ the API is required to make it easier to use from an application. Many times this ends up being an ‘adapter’ from the third-party API format to something more manageable and familiar to the application you are writing.

If you write this sort of thing in Java, you usually end up having a lot of duplicate code, specifically around error handling, that is hard to reuse without a lot of extra code. Lately, I have been using Groovy to solve these sorts of problems. The rest of the post shows a few ways I have learned to use Groovy to solve these problems in a better way. The result is simpler, more readable code.

For those not familiar with Groovy’s closures, let me explain what is happening here. The errorHandling variable is defined as a closure and is invoked around the code. This closure in turn takes a closure (function or code block in other languages) as a parameter. The error handling routine provides the stock error handling through the standard try catch syntax. Inside the try block, it invokes the closure that the user passed in. This allows the same error handling logic to be reused with the code to run being dynamically provided by the caller.

Groovy Categories for Mapping

Another problem is adapting between types in your application and types in the third party application. This is further complicated when dealing with collections of objects. Consider the following Java code to map from one system to another:

The groovy category essentially adds dynamic methods to both the InternalObject and ExternalObject. The definition of the category has static methods which take one or more parameters. The first parameter is the class that the method should be added to. When the new method is invoked, the object it is invoked on is always added as the first parameter.

Also being used here is the list ‘collect’ method. This replaces what needed to be done in Java with creating a new list, looping over each item, converting it and adding it to the new list. The collect method does this in a single step by using a closure that converts the object to a new format. The result is a new list with the objects in the new format but in a single line of code.

Putting it all Together

Using this category can also be combined with the error handling closure that was previously introduced. Consider the following updated version of the error handling closure.

With the previously defined error handling closure and category to handle the mappings, we can finally create a very minimal and readable adapter between our system and the third party system. Consider the following operation that uses the third party API. The input and output takes objects in the internal format.

Groovy’s closures allow error handling to be defined more easily and centralized in a single location where the code can be reused. Groovy’s categories allow data transformations to be done in a more readable fashion. The result is code that is easier to read and use.

I had the privilege of attending the SpringOne2gx conference this year in New Orleans. This is the first tech conference I have attended. Where I live, in Moncton, is a bit off the beaten path and is a bit far to travel to these types of conferences regularly. Past employers of mine have not always invested in their developers as much as they should. Thankfully, iWave Software was able to send me this year.

I had an excellent time at the conference and met a lot of smart and equally minded developers. There was plenty of great content at the conference and the Roosevelt hotel in New Orleans was a great place to host the conference. The food was amazing the the service top notch. On a whole the conference was a great experience.

For me, I made a Smörgåsbord of the conference taking a bit for all of the tracks that were provided. In the rest of this post I would like to cover some details on the plethora of technologies on display. Most of them were from SpringSource, the commercial company that backs the Spring technologies. There was a lot of great stuff but some of it I am a bit more skeptical on. The following are purely my opinions on what I saw. I am by no means an expert on any of these technologies but I do want to give you my first impressions of these from the perspective of a Software Developer and Architect working in the Enterprise IT space.

Spring Framework 3.0
One of the main focuses of the conference was the upcoming Spring Framework 3.0 release. Beyond just the framework itself, many of the other technologies leveraged some of the new 3.0 features. RC2 of the release should be available shortly. I would like to touch a bit on the feature set and particularly what excites me.

Spring Expression Language (SpEL) – This is one of the coolest new parts to the framework. A new expression language can be used to evaluate expressions related to other beans. In simple cases you can use something like #{refBean.value} in either the value attribute on a property or in the annotation directly. More complex expressions are also supported. This will be able to simplify configuration a great deal.

@Configuration – This is a rework of the Spring JavaConfig project that allows you to write your spring configurations in Java code directly. This project is now merged into the Spring framework 3.0. This feature did not excite me a whole lot. It seemed like something ‘cool’ but not something I would likely use. The main benefits of using this over XML were explained to be for refactoring, strong typing and losing the verbosity of XML. The refactoring and strong typing is less of an issue if you are using the Spring IDE (or STS) since it does all of that for you and even shows you where errors are. As for the syntax, I found the @Configuration to be clunky and still pretty verbose. If I wanted to express beans in code that are more readable, I would just use the Grails SpringBeanBuilder.

Java 5+ – Spring Framework 3.0 will be Java 5 and higher only. This is a great thing because it means that the core framework will fully support generics. All of the core libraries have been updated for this. In the past I stayed away from the JpaTemplate and other Spring libraries because the lack of generics made them more verbose than what I could write in wrappers of my own.

Task Executors – All of the scheduling stuff and executors were refactored. The Java 5 until.concurrent package is now used instead of the spring wrappers that they had. Also, scheduling is supported as part of Spring now. So for simple scheduling jobs (including cron), only spring will be required. This is nice since it is a common thing that is needed and Quartz is pretty heavy for this simple feature.

Rest Support – Spring Framework MVC now supports Rest. If you are an Spring MVC developer, this is very nice since you will be able to do REST directly from a @Controller method. If you are not using Spring MVC, there is no reason to look into this. The JAX-RS (JSR-311) spec is the way all of the other Java REST frameworks are going. I don’t see any reason to use this custom syntax over the standard for Rest unless you are using Spring MVC.

OXM – The Spring WS project created a bunch of stuff for handling Object to XML Mapping (OXM). Basically this is ORM for XML instead of a database. JAXB is one of the most popular frameworks for this and it is built in to the JVM. The OXM from Spring WS is now part of the core framework and allows anyone to have a single binding interface that can delegate to anyone of the many OXM frameworks out there including JAXB, XmlBeans, JiBX and Castor. This would be very nice for someone who wanted to support many different binding types but didn’t want to have to write all of that from scratch.

On a whole the Spring Framework 3.0 release looks solid. I am going to have to grab the RC and check it out.

Spring Roo
Spring Roo was one of the technologies being shown at the conference. I had heard of it before but did not really do much digging into what it was. In a nutshell, it is a coding by convention framework for building applications (mostly web) in Java. Similar to Rails or Grails, it provides a console based utility to create entity beans (model), controllers and views. Instead of taking the dynamic language approach, Aspect J is used to merge common tools and scaffolding into the Java beans you create. It is a very cool product especially if you don’t have the luxury of using dynamic languages.

One of the more impressive aspects of this was the console they created. The console was intelligent and had full auto-complete support. I would like to see other console based applications use this type of thing in the future.

I played around with this a bit. It worked well for what it did. It will likely be used mostly for rapid prototyping. I think if I had to do a new web project, I would probably just do it in grails since I am already comfortable with Groovy. This would be great for people learning Java or with a need to rapidly prototype something in Java using Spring. Also, it saves the hassle of setting up things like JPA and MVC. Once the scaffolding is in place, you can customize it how you need.

SpringSource Tool Suite 2.2 (STS for short)
The SpringSource Tool Suite is an eclipse based development environment for all things Spring. It includes the old Spring IDE capabilities for editing spring XML files, support for Spring Roo, Groovy, Grails and Maven. The whole package works very well (except for the fact that the 2.2 release did not include the groovy plugin).

Not much is really new here. I had been using the Spring IDE and Groovy plugins already. Most of the components are available standalone. This will be very useful for people who don’t have any of these plugins and provides a single download.

The main focus of the IDE was centered around delpoying to dm Server and tc Server, both of which are included. The new Groovy plugin is great and has greatly improved over the last little while. It is available standalone which is probably how I will use it for now as I don’t have a need for many of the other features at this time.

Spring Integration
Spring Integration is like an ESB inside a single JVM. It seemed like a good project if you had a need for this type of thing. For me, I see the crossing JVM boundaries (clustering) one of the most important aspects of an ESB and this does nothing to tackle that problem. The samples that were presented showed how the integration could be done with OSGi. The capabilities were there but the configurations were very verbose and required writing some code in places for filters and other such things. I kept thinking how much easier it would be to just do these things in code directly and they would be a whole lot simpler. Thread Pools are so much easier now with Java’s util.concurrent and dynamic languages do routing really well. Both of those I am comfortable with. I think this project still has a long way to go before it can truly compete with an ESB type product.

Spring Batch
Spring Batch was shown alongside Spring Integration for handling large batch operations. The framework looked good and provided a lot of monitor capabilities around the status of the batches.

SpringSource tc Server
tc Server is SpringSource’s enterprise version of Tomcat. Essentially they took standard Tomcat and wrapped it with better monitoring and clustered deployment capabilities. This looks like a great solution for services type companies who want to deploy to a more enterprise version of Tomcat. For a products company though, it may be more difficult to manage since it is a commercial product. Most customers I have encountered either want to use their millions of dollars JEE environment or expect you to provide your own.

SpringSource tc Server Developer Edition
The Developer Edition of tc Server was announced at this conference. The main feature included here was called Spring Insight. This is a very cool tool targeted at application developers. It provides real time monitoring of an application that gives developers information about what is going on in the application. It includes health information, performance and the ability to drill down on web requests to see each call. Even SQL queries being run are shown from this console.

Everything is done through AOP runtime weaving so no code changes are required to run in this environment. Also, having this free for developers means developers will have a great environment for analyzing the performance of their applications. This is definitely something to check out for anyone who has a .war application.

SpringSource dm Server
SpringSource dm Server is Spring’s OSGi server. Similar to tc Server, this is an enterprise grade OSGi Server. They provide a lot of features to make OSGi easier including being able to deploy a standard .war file to the server and have it run.

OSGi is ready for prime time and has proven to be. It is definitely not one size fits all though. It is not something you just use because you can or because it is cool. The demos of dm Server and OSGi with Spring with littered with caveats and workarounds. OSGi is a great platform but still has a lot of pitfalls for application developers. Yes it has some cool things you can do with dependencies and versioning but they are not free.

Grails
I did not get into any of the Grails sessions. There were too many sessions I wanted to attend and I have already had some exposure to Grails so I stayed away from these for the most part. On a whole, it seems like Grails has come a long way from a competitive web framework to one of the best ones out there. If I had a new web application project to do, I would be most likely to choose Grails to do it in.

Griffon
Griffon is one of the newer Groovy frameworks to come out. It provides a Coding by Convention framework for Swing development and is targeted for developing Rich Internet Applications. The core is based on the Grails and will likely be familiar for those who already have Grails projects. The framework is built around making MVC easy through Swing and use of data binding. I am really looking forward to digging into this more.

The biggest hurdle I see is being able to leverage this technology in an existing application. One of our main products is a very large Swing based IDE. The code has been around for a long time and could benefit from newer patterns for GUI development. I would love to be able to integrate Griffon into it but at this time it may not be easy. I intend to look into this more to see if it is possible.

Groovy
This groovy language has also come a long way. I started using it when 1.0 was released. Since then it has really grown in capabilities. I learned a few tricks this time that I was not aware of. Most of them had to do with the annotations that were added in 1.6.

The biggest thing for Groovy though was the IDE support. I love the groovy language but have found that I could write Java faster in many circumstances even though I write more than twice the amount of code. The reason is because of the IDE support. Code Completion, Import completion and Refactoring are huge. The new Groovy plugin for eclipse is fantastic and includes all of these. Finally, I feel I can switch to using Groovy as my primary development language on the JVM. This is also a big step toward convincing others of the benefits of Groovy.

I have been reading a lot lately about people hating on SOAP based web services. As a whole, the Web is moving more toward REST based APIs. This post is to make a case for WSDL and SOAP based web services.

Don’t get me wrong. I’m a huge fan of RESTful Web Services. I use them in many places where they make sense in the software I develop. I am not writing this post to say that SOAP/WSDL based web services are better than the REST style but I intend to point out some of the things that a WSDL does better.

It all comes down to perspective. Who is going to be consuming the Web Service? Is the consumer going to be a human or a machine? Let’s look at each case.

Humans Consuming Web Services. By humans consuming web services, I mean a programmer sits down and writes some code to use a web services. The common example of this is a developer from a website using the Flicker, Twitter, Google Maps or Facebook API to integrate with their site. Even in the business world, when a user needs to write a piece of code to connect two things together, REST is the clear winner. REST style web services are easier to work with and usually result in much cleaner code.

Machines Consuming Web Services. By machines consuming web services, I’m referring to software that interprets and leverages the web services. The primary examples for this are Business Process Management and Runbook Automation Software. This happens to be the area in which I develop software. In this space the goal is to allow machines to interpret the Web Services (or other technologies) and allow the user to just map from service to service. The user needs to know nothing about the transport or how the service function themselves as the machine is responsible for all of that. This type of software is typically used in the business world and not on the Web.

For this type of software, a strictly defined specification (a WSDL) provides a valuable tool to the software that needs to interpret it. A REST web service may be easier for a human to use by reading documentation, but we have yet to make a computer read documentation and produce results. Also, many REST style web services have XML formats that cannot be expressed in a Schema. This may be fine for a human, but XML Schemas provide an easy way for a computer to consume and understand the XML format. Yes, XML Schemas are very wordy and difficult to read by a human. They are also a pain to write properly. Even with their limitations, they do a good job of defining XML in a way that a machine can interpret them.

I see a lot of newer programmers who have only ever done work on the Web claim that REST is the best choice in all cases. We have graduated beyond SOAP to the superior technology. Even Joel Spolsky mentioned it on episode 64 of stackoverfow. REST as a superior technology is simply not true. REST may be the best choice for the Web but there are many other uses for Web Services besides the Web.

REST is getting closer to what WSDL has to offer. With WSDL version 2.0 or WADL you can define REST style web services. Maybe in a few years things will be different. Maybe we will get to the point where REST really is better than traditional web services. But we are not there yet.

One of the projects I am working on uses Apache Maven for doing builds. Usually, I let the Maven Release plugin take care of all of the version numbering. When I instruct maven to generate a release version or do a branch, it will ask for the new version for each of the artifacts in my project. The project I’m working on has about 15 different artifacts so switching the version numbers all of the time can take a bit of time. Allowing Maven to do this for me makes sense.

This brings me to my latest problem. We needed to change the version of all of our Maven artifacts outside of a release. Instead of incrementing the minor release number we are preparing for a major release (1.x to 2.x). I could do the version change the brute force way but being a programmer, I have strong instincts that prevent me from doing a repeatable monotonous task. Am I lazy? A bit, but I like to call it being efficient. I would rather take an hour and write a small script to do the task for me than take 15 minutes on the brute force approach.

What I needed to do was change all of the occurrences of 1.0-SNAPSHOT to 2.0-SNAPSHOT in all of my pom.xml files. Not only does this need to change for the main artifact but also for any dependencies.

My first attempt to write a script to do this was using Python. There were a few issues I ran into pretty quickly. I do not like most of the standard python libraries for XML parsing. The DOM library has the same issues as the Java DOM library. I just wanted a ‘simple’ XML library. So I decided to use the ElementTree library. I have used it in the past and it is very nice to use. I soon found that this would give me a few problems.

ElementTree does not support namespaces well. I wanted to use the findAll() method but it does not support namespaces.

Another thing I wanted to do is use full XPath. It is very simple to express finding all versions where the ../groupId = ‘mygroup’ in xpath. XPath seems better than writing this in code

Because of these issues I quickly decided that Groovy may be a better language for this script. Groovy has great XML support using their XmlParser. I actually produced a fully working script using the XmlParser that did exactly what I needed. The only issue with it was that when you write out the XML from Groovy after it was parsed, it was not maintaining the original formatting. I did not want to have the entire formatting of my pom.xml files be changed just for a version change. So this lead me back to XSLT.

I had done a lot of XSLT in my previous job and it really is the best tool for this kind of XML manipulations. It is very easy to preserve the original XML structure and make small modifications to it. The approach is to start with the standard identify stylesheet. This stylesheet essentially produces an exact copy of the original. From there you can add specializations to modify any things you need. Here is the identity stylesheet:

This 10 line XSLT did all of the processing on the pom.xml file I required. I wrapped this in a groovy script to walk the file tree and run it against all pom.xml files. This is the final script I came up with. It walks the tree for pom.xml files. For each file it finds any groups of com.chrisdail.project and replaces the version number with my new version number of 2.0.

I have written about the Groovy language a fewtimes. It is a wonderful language and integrates seamlessly with Java. Recently, I have found that it can be very useful for templates and filters.

Using Groovy for Filtering

Consider an application where there are many events that fire on certain conditions. The Observer pattern is often used for these types of event handlers. Also consider that the user (not the developer) wants to restrict or filter out events so only the ones they want are fired by the handler. Essentially the user needs a filter language where they can express their business rules. This filter needs to exist outside of the regular codebase so the user can change it on the fly.

Groovy solves this problem very neatly. Consider an Event class with the following properties:

severity

timesCalled

message

Assume that an instance of this class is provided as the object ‘e’. In groovy we could express a filter script that takes events with a severity less than 2 or times called greater than 5 as the following:

e.severity < 2 || e.timesCalled > 5

The above script is pretty simple and is easily written by a technical savvy user. I have written a simple helper class that facilitates creating reusable scripts like our filter script. Here is the code:

To use this script to actually execute the filter, you would do something like this:

// Creating the filter script. Assumes filterString is set by the user (I use spring for this)
ReusableScript filter = new ReusableScript();
filter.setScriptString(filterString);
// Typically this code would be running in the event handler. Assumes that an object e is available that is the event
filterScript = filter.newScript();
// Bind the 'e' object into the binding context of the script
filterScript.setProperty("e", e);
Object result = filterScript.run();
if (result instanceof Boolean && ((Boolean) result).booleanValue() == true) {
// The filter has passed.
// ... Code that performs the operation
}

Very little code is actually needed to put this together. The ReusableScript class can be used many times. The groovy code is only compiled once so performance is very good.

Using Groovy for Templating

A similar problem is the problem of templating. Consider an application that sends emails on error conditions. The user may wish to customize the email message that is sent out. They also may want to be able to use information from the application to provide meaning to the message. There are many template and expression languages available for Java. I have looked at using Apache Commons EL and Velocity. My main complaint on these languages is their complexity. It requires learning a new language syntax and integrating them often requires quite a bit of code. Let’s consider a more groovy solution.

Using the same ReusableScript class as used for filtering, we can provide templating. Consider a data class called Event that contains the following properties:

message

detail

hostname

Consider the following Groovy script that creates the body of an email message:

This is essentially making use of Groovy’s Templates that are built in to the language. The template engine can be used by itself standalone, but this provides a simpler integration. The other benefit is the if more complicated logic is required beyond a simple template, actual groovy code can be written beyond the simple script. The script is a full groovy script and would allow you to essentially do anything from it.

The template script would be loaded the same way as our previous filter script. This time instead of evaluating the script as a boolean, we get it as a string.