The Music of Time

This is not, unfortunately, a blog dedicated to that great twelve volume series of novels by Anthony Powell. However, it is a form of record of my journey through the world of information technology; a journey motivated by both profit (it is intimately related to my profession) and inquisitiveness (I love to know how things work).
It is my intention, therefore to record. These records are primarily for my benefit, but may be useful to others as a side effect.

Thursday, October 04, 2012

Thanks to Russ Hart for providing the info on how to get this to work. I just cut and paste, and then blogged it.

It's nice to write unit tests. I wanted to write a set of tests for a method that I'd marked with the Spring @Transactional annotation:

@Transactional(propagation = Propagation.MANDATORY, rollbackFor = {MessagingException.class}) @Transformer public IncomingEmailDTO receiveMessage(Message message) throws MessagingException { IncomingEmailDTO emailDTO = null; ...The tests for the method were simple. But then I realised I wanted to also test that rollbacks were happening or not as required as specified by the "rollbackFor = {MessagingException.class}" part of the annotation.A quick aside (because I was asked this by the person who gave me the solution to this, and it's entirely valid). Why did I want to test this? Aren't I just testing that this Spring annotation works? In part, yes, this will be the effect of any tests for this. But there was something else I wanted. Unit tests, over time, build into a massive, executable spec. for your system. It would be very easy for someone in the future to change this small part of the annotation (or remove it altogether) and have a very significant effect on the running of the whole system. Consequently, by adding tests which check that this test is a) transactional, and b) set to rollback for specific exceptions only; I can protect myself against this unfortunate outcome.

Tuesday, November 02, 2010

The @Debt annotation can be used in Java 6 or above to measure in code instances of technical debt. If a configurable threshold is exceeded then the build will fail.

It is intended to be used by both developers and dev leads. The number one use case is the situation when expediency has driven you to make a design or implementation decision which you are not 100% happy with. Normally you will moan to yourself quietly, write the less-than-pretty code, and move on. If you are lucky you will remember the exact position where this trade-off was cast in stone; but probably you will not. Then in the future, you hit the problem again, and again there is no time to put in a "nicer" implementation. You grumble again, and move on. If only there was a way to mark all the occurrences of this technical debt, and to indicate every time it later causes you real development pain.

The @Debt annotation lets you do this. Now, when you make the concession, you quickly add the "@Debt" to the member in question (adding a quick description string and set the counter to "1") and move on. Later on, when you hit it again, you just increment the counter. Meanwhile, behind the scenes, at every build you are automatically trawling through your codebase and crunching all this debt and keeping a track of it. How debt-ridden is a given part of your code? Now more than relying on smells alone, you can have a look at the @Debt output and have a real idea.

But beware! You need to be disciplined in your marking and updates of your @Debt trackers. Otherwise it's as useless as thoise unit tests which now fail so you switched off. You have been warned!

Thursday, October 28, 2010

As I said in my previous post, our project has suddenly taken a new path. There are two bits of collateral which I'd recently produced which will now no longer be required. Here are the notes for myself, and anyone else who might be intetested.

Our project has suddenly taken a new path. There are two bits of collateral which I'd recently produced which will now no longer be required. Here are the notes for myself, and anyone else who might be intetested.

Part (i) - Way of Working for an Agile Distributed Team

3 week iterations (all teams starting and ending on the same days)

Sprint Planning with the teams (Planning Poker for Stories, hours for tasks)

Daily Scrums (3 questions plus "What have your learned?")

Prioritised Backlog (Stories for the next-up phases, Use Cases for further out, Epics beyond that)

Product Owners own Product Backlog

Release Plan (always visible with named internal releases each iteration)

I've had some time recently and managed to get back to looking at my Hudson RTC plugin. I found a lot of things making the 1.0.0 version really unstable and have made some updates and come up with 1.0.1.

To get it and start getting the Hudson build love in your RTC projects visit the project site.

Please let me know too how it works for you - the site has an issue tracker so please log all bugs and RFE's there.

There are some extra features I'm working on for 2.0.0 which hopefully will be along soon including change set parsing (let Hudson know just who's changes sparked a build), personal builds, and automated releases. Watch this space...

Friday, May 28, 2010

I'm in the midst of transitioning out of my role as a CM/CI/Release Manager. I've learned a lot while doing it (some things the easy way, some the hard way), and thought I'd blog my conclusions and lessons learned so I don't make the same mistakes then next time.

Grow; GraduallyThere is immense value in even a small bit of SCM, build automation, or CI feedback. As long as your approach is flexible (and if your tools don't support this then seriously consider changing your tools).

What does this mean in practice?

For SCM, choose a tool, (a free one preferably as this is quicker), get it installed, and get everyone checking in their code and other artefacts early. In my experience a distributed SCM (i.e.Git or Mercurial, or even Jazz SCM) is best as it's structure can easily be changed as the project changes over time.

For build automation, choose anything (Ant, Maven, Rake, hell even shell scripts) but get it set up at the start and have all your bits use it. Also seperate the build from the environment right from the start. This makes it very flexible.

For CI choose Hudson (IMHO it's the best engine at the moment by a country mile). Get it set up on a dedicated server right away (this will take 30 minutes) and get it polling your SCM, running builds, and going all red/yellow/green right from the outset.

Start with a Small, Simple, Core Build as Early as PossibleContinuing on from the last tip, you want to depend upon Hudson from the outset. Get the team aware of the benefit of the feedback it provides. Make it the team's friend, rather than their enemy.

To do this resist the urge to get clever to begin with. A single build will do at the begnning. You can grow it later. You'll read all about upstream and downstream builds and promotion and everything else. These are all great concepts, and you will use them, but nt yet. Implement them when the time investment warrants it. At the outset, your code base will be tiny, and you will have few Unit Tests. Don't worry about your build speed. It's not slow yet. Just get the code compiling, and the unit tests running.Ensure the Whole Dev Team Understands how their SCM Tool WorksYour SCM is a powerful tool. If you understand it beyond the basic "checkin/checkout" dynamics it will allow you to have maximum confidence in the code you are working on both individually and as a team.

On our project we were soon taking advantage of the facilities for personal revision history, suspension and revision of change sets, creation of patches, reversals, and more. To get to this we created a training session which was run for each new joiner which took them slowly through the basics of the Jazz SCM "Change Set" system, and then build upon this so they all had a workable meta model of their changes in the context of their team, and the wider project.

Beyond this (and as a result of having to evolve the SCM model from the simple one we started with to the one we ended up with when everything went offshore) the CM/CI team had a deep understanding of how the entire system worked, which allowed us to do some really clever tricks, digging people out of messes when they did (infrequently) get into them, and making sure we could have just the right amount of seperation between bugfix, next release, and PoC / Investigation development.

Have a SandpitAfter the basic setup for both your SCM and your CI build is done you'll want to get cleverer and cleverer. This isn't a bad thing, but you will make mistakes. Especially if you want to get really clever.

We made the mistake of doing this experimentation on our "production" systems (i.e. the SCM and CI servers that the Dev Teams were using. Aside from the danger of catastrophic problems which you may cause, there is also the confusion you generate. CI is best when it is simple, clear and reliable. Red is bad. Green is good. You don't want to have to tell people to "ignore this for the minute while we get it working" or "don't worry about that, it'll work soon." This errodes trust in your tools, and gives others a source of excuses. As the CM/CI team, you need to lead by example. Hide your mistakes. Have a sandpit.

Make Sure the Architecture is "Developable"What do I mean by "Developable"? It's my first real neologism and one I'm more and more confident of the more I work in Software Development. In reality it needs a whole post to itself (perhaps even a book), but in essence I mean that the chosen architecture and component technologies which it comprises should be easy to develop against. Perhaps a few examples are required.

Compilable: Is it straightforward to compile your code? Do I just need the compiler, the dependencies, and the build scripts? No? I need the vendors too installed, and their server, and I need to deploy a dependency to compile downstream projects? Think again...

Unit Testable: Is the code you write easy to unit test? Can you automate the running of these tests with your IDE and CI server and collect and view the results effectively? Is it easy to mock? No? I need to deploy in order to unit test? Perhaps you should re-consider...

Tool Independent: Can you run your builds headlessly, without the need to install an IDE? No? Is this the 1980's? Come on...

Version Controlable: Can we easily manage the code and configuration files in an SCM? You don't know SCM is? Let me explain...

Manageable Dependencies: Can we automate the management of dependencies? You want me to check-in the compiled results? Are you serious? Alarm bells...

Quick to Build: Can I poke my build scripts (on my machine, or via Hudson) and within a relatively sensible period of time get a red/green feedback? It takes 30 minutes just to compile, and its going to get longer as you add more code? Hmmmm

Quick-Turnaround, Lightweight Dev Environment - Can I deploy my latest edits to my local server and see the results in a few minutes max? Can I develop on the same platform as production without having to take up hundereds of gig's of disk and have a 64-bit OS to address all the memory required to run all the servers and databases just to compile and run the unit tests? No? Have you heard of Tomcat?...

I'd recommend you think about (and investgate if needed) all these things and more when you get told what your architecture is and what you're going to build it in. Pushing back at the start is a lot easier than pushing back later on. Trust me...

Publicise the CI BuildDespite the fact the Hudson is a superb resource which all developers should embrace because it makes their lives easier, most of them hate it because it goes red when they screw up. In an ideal world they would love to know that the change they just made caused a problem, but the majority seem to want to forget about it until later, when the detail has been swapped out of their mental L1 cachee and it's far more of a challenge to resolve.

You can remedy this by making Hudson and the status of Jobs ubiquitous. Get Hudson to send emails; use the Hudson Tracked and Growl to pop up toast on their screens (and play a sound too); but best of all set up a build radiator. At worst, this should be a screen, widely visible to all (especially the Project Managers / clients) and displaying the colours of all key jobs. (All jobs preferably). At best, it should be a singing-dancing, attention grabbing machine which rewards good behavious and punishes the bad. Come on, get creative!

Automate the Release Process as Much as Possible - And Keep it as Simple as PossibleMy final lesson learned is around making releases. You want to be able to make releases as quickly as possible, with as little manual intervention as you can get away with. I'm including the packaging, documentation, and deployment of these releases in this.

The aim is to make it easy to release super-frequently. Hudson isn't your last line of quality feedback. There are the testers too remember. The more you can give releases to them, and the quicker you can give them an update with all the fixes the more time they'll be able to sit about testing and increasing the quality of your product even more. It is all about feedback and quality after all...

Sunday, February 07, 2010

IntroductionIn my previous post, I introduced version 1.0.0 of Cowley, a Hudson plugin for IBM's Rational Team Concert (RTC) SCM system. I'd love for the plugin to be a simple one-click install like many others in the ecosystem, but unfortunately it currently relies on the RTC Build System Toolkit which is freely available, but isn't in any public Maven repositories. This means, in order to simply use the plugin, or to build it from source, you need to do a little Maven / Ant / Jar magic. This post in the series will tell you how.

Note: This post is specifically geared towards getting the required pieces of the Build Toolkit into a Maven-usable state. However, there is nothing here which couldn't be taken and reused in many other circumstances.

Pre-RequisitesI'm assuming for the rest of the instructions that you have Java 1.6 and Maven 2.x installed. If not, why not? Get to it!

Obtaining the RTC Build System ToolkitJob 1 is to get the Jazz RTC Build System Toolkit. You can download it from jazz.net (Windows) (Linux) (registration required).

NOTE: The rest of the instructions are based on the Windows version of the toolkit. I guess, as its all Java based, it's really similar. Post comments on this blog if there is additional info I need to add here.

Once you have the Build System Toolkit downloaded, unzip it somewhere temporary.

The Maven / Ant / Jar MagicNext you need to get all the jars which come with the IBM Toolkit, wrap them up as a single jar, and add it to Maven for quick access and reuse. To do this, you'll need the following maven pom.xml. Cut and paste it into a file for use.

Now if you create a directory, put this new pom.xml file into it, edit the property "rtc.build.toolkit.plugins.dir" property and run the "mvn install" command. This will get the maven plugins and dependencies required, create your uber-jar, and "install" (i.e. copy) it into your local maven repository.

Congratulations!You now have the required elements of the RTC Build Toolkit installed in maven, ready to either be added direct to your Hudson install to enable your Cowley plugin, or so you can compile my RTC Proxy API from source.

Either way, look out for the later posts in this series to find out how to obtain and use the Cowley Plugin without looking at any more code, as well as the gory details of the plugin development (if you're feeling sadistic).

Saturday, January 23, 2010

There is a tantalising set of documents on the OpenOffice.org wiki about how you can use the officebean.jar which comes with all OpenOffice installations to display documents in an applet. The problem is, its out of date, and the link to the source is dead. But don't let that stop you...

Netbeans (I used 6.8 - get it here. You could use another IDE if you like)

The CodeFirstly I created a new Java Class Library project in Netbeans called OOoApplet. To this I added a single class which I called "OOoBeanViewer.java". I then (lazily) went looking for the example code. I found it on Koders.com here. I cut and pasted this into my OOoBeanViewer.java stub in Netbeans.

The DependenciesIn order for this to compile and run I had to add the following dependencies:

The EnvironmentBecause Open Office doesn't really run in the applet (rather we start it, and then redirect its display to the applet) we need so set things up so that the OOoBean knows where to look. This wa the hardest part as the assumption in the docs I could find was that this demo should just work. It didn't for me.

To get it to work I needed to add a Windows environment variable called UNO_PATH and set its value to be the path to the program directory of your OpenOffice install. I set mine to: "C:\Program Files\OpenOffice.org 3\program"

To get this to stick I then had to reboot. A check "echo %UNO_PATH%" at the command line showed that this had worked.

Running ItAll that was left was to compile and run. This was done simply using Netbeans. The result is a mostly empty Applet window with some buttons down the side. Fear not! You need to create a new document to see what you desire. This is what you get if you select "New Document ... > Text Document":