Category: Apache

In Part I, Information Radiators, I covered what they are, what the main benefits are, and the approach I usually use to set them up. This post goes in to more technical detail on how I extract this data from Jenkins.

The Jenkins XML API is very useful for automating tasks like this – if you simply append “/api/xml” to a
Jenkins job URL, it will serve up an XML version – note there is also a JSON API and a CLI and plenty of other options, but I’m using what suits me.

The Jenkins XML API

For example, if you go to one of your Jenkins jobs and add /api/xml like this:

e.g. “true” or “false” – this was very useful and easy to do, but the functionality was removed/disabled in recent versions of Jenkins for security reasons, meaning that my processes that used it needed rewritten 🙁

Extracting the data – Plan B…

So, here’s the new solution I went for – the real scripts/methods do some error handling and cleaning up etc but I’m just highlighting the main functions and the high level logic behind each of them here;

get_url’s method:

query a table in MySQL that contains a list of the job names and URL’s to monitor
for each $JOB_NAME found, it calls the get_file method, passing that the URL as a parameter.

get_file method:

this takes a URL param, and uses curl to fetch and save the XML data from that URL to a temporary file (“xmlfile”):

curl -sL "$1" | xmllint --format - > xmlfile

Note I’m using “xmllint –format” there to nicely format the XML data, which makes processing it later much easier.

get_data method:

this first calls “get_if_building” (see below) to see if the job is currently running or not, then it does:

I then have JSP pages that read data from that table, and translate things like true/false in to HTML that sets the background colours (Red, Amber, Green), and shows the appropriate blocks and details per job.

If you have a few browsers/TV’s or Monitors showing these strategically placed around the office, developers get rapid feedback on the result of their code changes which speeds up development, increases quality and reduces development time and costs – and they can be fun to watch and set up too 🙂

Information Radiators are used to provide people with feedback on the current status of code builds and automated tests in Continuous Integration and Agile development environments.

The basic idea is that when developers commit a code change, they can easily and quickly see that it has been picked up by the automated build process, and then (ideally within 10 minutes) see the result of their change; did the build succeeded and did the automated tests pass?

Martin Fowler’s description goes in to more detail on the ideas behind this approach and the function that Information Radiators serve.

The normal convention for these is to use colour coded blocks per build, using:

Green for good/passing jobs

Amber for either currently building/running jobs (or sometimes for unstable ones)

Red for failed jobs

Generally you want to keep things as clean, simple and uncluttered as possible, but sometimes it is helpful to add in a bit more info.

Details I have occasionally found worth adding include things like;

name and/or picture of the user who triggered (or broke!) the build

commit message from the code change that triggered the build

build number

history – number of recent fails or passes

date/time last failed and last checked

if you use amber for “unstable” rather than “build in progress”, you may want to add text to say if the job is currently building – I often use the “spinning wheel” icon thingy from jenkins itself :

Why build your own?

There are tons of readily available plugins that allow you to quickly and easily produce a Radiator or Wall panel from Jenkins, so why go to the bother of making your own?

Plugins are usually linked to one Jenkins instance (the one they are running on) and I have often found that alone to be too restrictive – having too many different radiators all over the place makes things too cluttered and uncertain, and people can easily start to “switch off” from them all – having one screen that people can understand at a glance usually works best.

Changing requirements – developers are constantly wanting/looking to improve processes and often come up with ideas and requests for things to try that may help them do their jobs – adding a bit of information from another source, for example, or changing the colours used to a different shade, or adding curved borders etc etc…

What I have found often works best, is to get all of the data I am interested in inside a database then write my own simple but flexible presentation layer from scratch – this gives me all the flexibility I could want (or may find myself wanting or needing later…) and importantly, it also allows me to leverage additional benefits by combining data from Jenkins with data from elsewhere – this means I can easily produce reports, charts, metrics etc that present a view comprised from multiple data sources throughout an organisation – for example, you can then easily create reports that combine:

version control – the actual code change can be extracted from version control and linked to both the developer and bug details – the “svn log” command is useful for this

environment monitors – state and health of environments; database and app server health, deployed patch and code levels etc (again, these are usually other Jenkins jobs!)

and you can add in anything else you can get your hands on 🙂

This allows you to track the flow of a change right through the development life cycle – from the initial change/requirement/defect to the change itself at the code/file level, then the testing and building of it and the eventual release. This is far more than you need for a simple Information Radiator, but using this approach means that you can easily reuse much of the work in different ways.

About 10 years ago, I bought a SunRay 1g Client box with the plan of setting it up as a thin client to the (physical and very noisy) Sun Solaris Server I was running at the time. The old Sun server sadly didn’t last much longer, and I replaced it with a FreeBSD one before I managed to get the SunRay up and running… so it sat in a cupboard until recently.

The new plan is to use the SunRay 1g as a thin client to a Solaris VM running on my ESXi5 server. This seems like a good idea as it gives me a way to access and manage the ESX server without having to fire up another machine – plus, I think SunRay clients are awesome bits of kit – if I didn’t have an iMac I would use these, as they are silent, boot instantly, use hardly any power and make no noise… it seems like the ideal compliment to an ESX server.

Here’s a pic of my SunRay client…

So, here are some notes, links and pics on installing an Oracle Solaris 10 Virtual Machine on ESXi5, setting up the SunRay Server Software, and getting the SunRay client (finally) working…

From the Oracle site, Sun Ray Clients are:

“… simple, low-cost devices that are ideal for displaying server-hosted virtual desktops. With no moving parts and no local operating system to manage, Sun Ray Clients provide a cost-effective, highly functional thin client alternative to desktop and laptop computers and reduce many of the problems associated with traditional desktop deployments.”

They need a Server running the SunRay Server Software to connect to – I think you can use Linux these days but I wanted to set up a Solaris host anyway, so I went with the traditional approach – albeit running on VM Ware.

None of this was very difficult, and the only slight issue I had was getting DHCP set up correctly; as I understand it, Sun Ray clients need to be sent additional information when they get an IP address, and that means it’s easier to set up the SunRay Server as your DHCP server – once I’d done that (and configured my Router to act as a DHCP Relay, handing DHCP requests over to the Solaris host), it all worked well.

Install Solaris 10 on VMWare ESXi5:

I downloaded the DVD image from Oracle, and then uploaded it to the data store on the ESX Server, created a new VM and set it to mount the image as a DVD drive at boot time. The spec of my Solaris 10 VM is 8GB RAM, 2x vCPU and 500G disk for now – I will change this later when I see how it performs.

This command starts up the (GUI) DHCP Manager wizardy thing, which will prompt you for all required info to set up DHCP:
/usr/sadm/admin/bin/dhcpmgr &

As as I understand it, your SunRay clients will look for a DHCP server to (also) give them the details of the SunRay Server to connect to. I coudln’t see a way to use my existing DHCP server (my broadband router) and still be able to tell the clients how to connect to the Solaris Server – that would have been preferable for me.

cp /etc/nsswitch.dns /etc/nsswitch.conf

add the google DNS Server and the internal IP of my router to /etc/resolv.conf….

Once that was all done and my Router was configured to relay DHCP requests to the DHCP Server running on the Solaris 10 VM, I was able to switch on my SunRay 1g client and be presented with a logon screen – yay!

And finally, here’s a pic of the SunRay client sitting on top of the HP ProLiant ML110 G6 ESXi5 home lab server, which is running the Solaris 10 Virtual Machine it’s connecting to…

Here I am adding the first two scripts with chmod 755, then adding everything else which will get the default/umask permissions. I exclude the previous files – not sure if that’s required or not but I don’t want to risk overwriting them.

Now when you gunzip and tar xvf the resulting build, you get the required permissions.

Here’s a note of what I needed to do in order to get WordPress serving as the default site on my domain – it was originally at www.donaldsimpson.co.uk/wordpress/ and I wanted it to just be www.donaldsimpson.co.uk

A bit of a Google shows there are many ways to do this, but here’s how I did it:

vi /opt/bitnami/apache2/conf/httpd.conf

then comment the current entry and add a new one pointing to the htdocs dir for WordPress:

Then restart Apache (/opt/bitnami/apache2/bin/apachectl restart or similar) after which you just need to go to the WordPress Admin General Settings page and change these values to point to the root of your site/domain:

WodPress address (URL) www.donaldsimpson.co.uk

Site address (URL) www.donaldsimpson.co.uk

And that should be that – you can now delete that backup you made at the start…

Update:

It’s may be a good idea to define your WP_HOME and WP_SITEURL in your wp-config.php file too, like so: