Atanas Hristov

Friday, August 26, 2011

Hudson is a tool for continuous build, integration and deployment widely used in the Java world. Hudson can be used also with other platforms and programming languages and on many operating systems. A lot of plug-ins are available for Hudson.

For the Dot.NET world Hudson provides complete environment for continuous integration. The tool compile code with MSBuild, can run NAnt scripts, can run NUnit tests, check for code rules with FxCop or Gendarme, create documentation with Sandcastle.

The Hudson server by default provides integration with Subversion. It can be installed on Windows to run as a windows service.

The configuration of the jobs to run on the Hudson server is done completely from the web-based console of the tool. Also from there we install plug-ins.

Installing Hudson: Prerequisites

Installing Hudson on Windows is easy task. Following are the steps we needed in our case.

In this scenario we'll be using Hudson for Dot.NET development in conunction with the following tools: MSBuild, NUnit, FxCop, Gendarme, Sandcastle, so that we'll build, run unit tests, check code styles and create API documentation for our projects.

I will demonstrate using Subversion for source control system. But with Hudson you can easily use other version control systems too.

Install Java. Hudson is a Java application.

Download Link: http://www.java.com/en/download/ie_manual.jsp

Add to the system environment variable %PATH% the directory name where the java.exe is:

"C:\Program Files (x86)\Java\jre6\bin"

Open new command window and try to check the version of Java installed:

In our case the installer suggested by default the following install directory:

C:\Program Files (x86)\Sandcastle\

Download and install the Microsoft HTML Help Workshop from:
http://go.microsoft.com/fwlink/?LinkId=14188

Run the downloaded file htmlhelp.exe and follow the instructions.

The default installation path was in our case:

C:\Program Files (x86)\HTML Help Workshop

Download and install the Sandcastle Help File Builder from:
http://shfb.codeplex.com/

Unpack the downloaded zip file and run the SandcastleInstaller.exe installer application. It comes with nice wizard GUI. Follow the instructions. The installer will ask you to download and install for you all the additional applications it will need in order to work. Just answer with "Yes" on those questions.

You will skip the installation of Microsoft Help 2, this is not a necessary part to have.

It also will ask you to apply a patch to Sandcastle. Answer with "Apply Patch" on this screen.

If you wish you can install also language pack for German language.

Optionally it will download for you the MAML guide - the guide from Microsoft for the documentation markup language, web project custom providers - a package with information for producing XML documents for web applications and website projects, a HTML to MAML converter.

If you don't have Visual Studio installed skip the section :MAML Schema InteliSense"

After this step it will suggest to install for you the SHFB - Sancastle Help File Builder. Cilck on "Install SHFB" on this screen.

The SHFB in our case was installed into:

C:\Program Files (x86)\EWSoftware\Sandcastle Help File Builder\

On the last screen of the wizard click on the 'Reboot' in order all the changes to take effect on your system.

Visual Studio Templates

If on the machine where Hudson will be running the visual studio is not installed you may need to come some Visual template files from developer's machine. For example copy to the machine the complete directory:

To easily integrate a Visual Studio Project into Hudson, I would suggest the following project structure and files to be created.

Suppose we've got a project named "WebAPI"

In my project directories in Subversion I would prefer having structure like:

directory .\References - copy here all the third party DLLs you'll be referencing from the project

directory .\WebAPI - here is the project with the production code: a Visual Studio project named like the directory - WebAPI.csproj

From project properties in the Visual Studio, in section "Build" go to sub-section "Output" and enable check the box "XML documentation file". The name of the output file in "Debug" target build configuration should be the same as the specified in the Sandcastle project (below).

directory .\WebAPITests - contains the project with the unit tests for the WebAPI project: another Visual Studio project with Visual Studio project file named WebAPITests.csproj

The project with the unit tests references the project with the production code.

file WebAPI.sln - the Visual Studio solution file.

file WebAPI.shfbproj - the Sandcaslte project file

To create the Sandcastle project file run the "Sandcastle Help File Builder GUI" application. From the project properties the make the HelpFileFormat to be "HtmlHeml 1, Website".

Documentation sources are:

WebAPI\bin\Debug\WebAPI.dll
WebAPI\bin\Debug\WebAPI.XML

which is the output after compiling the WebAPI.csproj with target build configuration "Debug".

note the OutputPath from the "Project Properties" panel. The output goes to:

.\Help\

This is the directory to which we'll make link in Hudson for the API documentation of our project once the documentation is generated. That way we'll publish the generated from the continuous integration API documentation as a web-site.

file run-fxcop.bat - a batch command to run FxCop from Hudson. Example:

note we specify the MSBuild from the version 4.0 of Dot.NET and as a project file for the build project we point to the file WebAPI.shfbproj

And here we've got structure similar to this:

All the output xml files from the batch commands will be used by Hudson to show us results of running unit tests, checking code style, etc.

Installing Hudson: Installation

At this point we are ready to install the Hudson. We'll install Hudson as a Windows service on Windows.

Prepare working directory

Create directory

C:\Hudson

In this directory Hudson will be installed. From the Windows control panel set up system environment variable:

HUDSON_HOME=C:\Hudson

Open new command prompt and test the new variable:

C:\>echo %HUDSON_HOME%
C:\Hudson

In this directory Hudson will check out from Subversion our code and will run the jobs to compile and run unit tests. So, this will be the Hudson's working directory. There Hudson will accumulate files with historical information.

Install Hudson

Create directory where to install the Hudson server:

C:\Program Files (x86)\Hudson

Download into this directory the latest version of Hudson from:
http://java.net/projects/hudson/downloads/download/war/hudson-2.1.0.war

Note: you can always upgrade Hudson automatically to the latest version directly from the Hudson's web console - after the Hudson installed as windows service.

Rename the file as removing the version information from the file name. Check the install directory of the Hudson server:

Scroll down the page and click on "Install". After Hudson installed all those plugins, click on "Restart when no jobs are running".

Go to Hudson\Manage Hudson\Pugin Manager, open tab Installed and check if all these Plugins are installed:

Go to Hudson\Manage Hudson and click on "Configure System".
Scroll down to "MSBuild Builder" and click on "Add".

Write for name "MSBuild Dot.NET 4.0".
Write for path the full path name of the msbuild.exe file for Dot.NET 4.0, in our case it was

"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe"

You can add for other Dot.NET frameworks if you need. Later from the concrete jobs you'll use them as appropriate in your case.

Scroll down, click "Save".

Configure Hudson: Add Jobs

Now we can add our project for continuous integration as a job on the Hudson server.

Continuous build

This is a build which will be checking every minute for changes in the Subversion repository, if they are some changes will check out the latest changes, build our project and run the unit tests.

Click on Hudson\New job. Set the name of the job, in this example "WebAPI continuous build" and select the option "Build a free-style software project". Click on "OK".

In section "Source Code Management" select "Subversion". Set the Repository URL as appropriate.

Move the focus to another field. Hudson will try to access your Subversion server and if the authentication failed will show red colored message next to the text box with the repositiry URL. The message should contain an html link "enter credentials". Click the link and follow the instructions. Eventually you'll be able to give your user name and password if you choose this form of authentication.

Once you successfully finish this, you'll come back to the previous page.

Click on Save to save the job. In short time Hudson should start a build for this job. If the build failed, check the "Console Output" from the job menus.

Note: you can at any time schedule run for a job. Go to the home page of the tool, click on the link to the job "WebAPI continuous build", from the left-side menus click on "Build now".

You can check the result of your unit tests from the link "Latest Test Result". Also you can browse your source code tree from the "Workspace" link and check for latest changes from the link "Resent Changes".

Now that we added a build project we've got nice dashboard on the home page of our Hudson server:

Extended build

This is a build which will run once in a day and will run code-style checks and will create documentation for the API.

Click on Hudson\New job. Set the name of the job, in this example "WebAPI extended build" and select the option "Build a free-style software project". Click on "OK".

In section "Source Code Management" select "Subversion". Set the Repository URL as appropriate.

Move the focus to another field. Hudson will try to access your Subversion server and if the authentication failed will show red colored message next to the text box with the repositiry URL. The message should contain an html link "enter credentials". Click the link and follow the instructions. Eventually you'll be able to give your user name and password if you choose this form of authentication. Once you successfully finish this, you'll come back to the previous page.

In section "Build Triggers" select "Build periodically" and specify "40 02 * * *" - just an example - to run the job once a day at 02:40 PM.

In section "Buld" click on "Add build step", then "Build a Visual Studio project or solution using MSBuild". Select/write (we are following the example file and VS project names from above):

Thursday, February 3, 2011

In my last post I created small java script function that finds the size of image by URL. I used YUI 3 test framework for doing unit tests while I worked on that function.

Because YUI 3 test framework is in-browser framework we are facing the challenge how to automate our unit tests and how to apply TDD with automated continuous build and automated continuous integration. As we have to manually re-run tests it is a challenge to keep testing our java script on several web browsers on different platforms.

jsTestDriver is automation tool for unit testing for java script. It is a server written in Java. From several several browsers you open URL hosted on the jsTestDriver server. The jsTestDriver can then automatically re-run the unit tests whenever they are some changes in the source code and thereof shows if the unit tests failed or passed on the currently connected web browsers. That way you can test as many as you like combinations of browsers and platforms in an automated way.

I'll be working on Linux for the examples, but the jsTestDriver works on Windows too. Also, it is always a good idea to develop your java script libraries outside your web projects and ideally to use in your web projects already tested, optimized and minified java script file.

The installation of jsTestDriver is very simple. Download the jar file from the project's web site:
http://js-test-driver.googlecode.com/files/JsTestDriver-1.2.2.jar

Just copy the jar file on directory where it is convenient for you. Here is a snipped from my directory structure:

You should export environment variable $JSTESTDRIVER_HOME (%JSTESTDRIVER_HOME% on Windows, go to My Computer -> Properties -> System -> Advanced and change the Environment Variables to make the changes permanent).

As I mentioned in my last postbuild you can ignore, I normally have build directory where I would generate minified, tested java script - the stuff that I'll be using in my web project. We don't need the deps folder this time, as we will not have any dependencies on 3rd party java script.

The file jsTestDriver.conf is a configuration file for the jsTestDriver with the following content:

It states, we'll run the jsTestDriver on port 4224, the server will load all the java script files from directories src and test. All the load paths are relative to the directory where the jsTestDriver.conf file is. Please note that the order the directories and file names are important. In our case the jsTestDriver will first load the source code java script files, then the unit test java script files. More information for the jsTestDriver configuration file you can find on the projects wiki page:

http://code.google.com/p/js-test-driver/wiki/ConfigurationFile

src - this directory holds just one java script file in which is the source code of function ImgSize()

The queue parameter accepts inline function via method add(), the function will run whenever the browser downloaded the image and calculated the size of the image.

As you can see, the image file should be accessible from URL from the testing browsers, so I just put the gif image on web server on the following URL above. Keep in mind, you have to adjust that the way you need it.

The syntax is very similar to YUI 3 test and even simpler. Whenever the browser loaded the image, we are going to check if the ImgSize() function gets the proper image size information. Please note that we are not dealing with DOM and UI. If you want to test DOM operations you can add HTML to the unit test files, for more information check the wiki pages of the jsTestDriver project:

Now, how to repeat the tests automatically whenever the source code of our java script changed?

If you have Ruby installed on your system, you can install the jstdutil package:

gem install jstdutil

It provides a wrapper to jsTestDriver so instead of writing every time this long command lines on the shell, you just - from the directory where the jsTestDriver.conf file is run:

ahristov@medion:~/Desktop/Projects/js/proj/play$ jsautotest

Every time we do changes on our java script source code the jsTestDriver will automatically start doing the unit tests and then will report the result:

Let's change the test, make the test fail, change the following line:

assertEquals( 38, arguments[0][1]);

to

assertEquals( 39, arguments[0][1]);

as the jsautotest is running. At the moment we save the javascript file, jsautotest automatically re-runs the tests sending the appropriate command to the jsTestDriver server and all currently captured browsers start running the tests. We get error in our tests and the jsautotest console explains this with output similar to:

Changing the assert back from 39 to 38 in the ImgSize-test.js makes jsautotest again to re-run the tests on the jsTestDr4iver server, and this time they successfully pass.

The jsTestDriver project has the following URL:
http://code.google.com/p/js-test-driver/

The home page of the jstdutil Ruby package:
http://cjohansen.no/en/javascript/jstdutil_a_ruby_wrapper_over_jstestdriver