Year: 2013

The year 2014 is almost here so it’s time to take a look back and make some plans for the coming year. In 2013 the blog was a bit more active than previous years and I managed to put together 13 articles as I last year promised. Not one article per month but on average :) So let’s have a quick look what I wrote.

Looking back

This year it was my third visit to Munich and Fujitsu Forum. With keynotes and breakout sessions the forum provided a view to human centric intelligent society and how the future might look like with technology trends like Internet of things. It was again nice to see other bloggers and the Fujitsu team, get some insights to ICT and not to forget the hands-ons with new LIFEBOOKS like U904 and T904 and new STYLISTIC tablets.

On software development I wrote on three main topics: testing, monitoring and documentation. In software development the code is one part but also it’s quality and testing it are also essential aspects to make it great. I learned some new tricks with testing Java EE applications and wrote how to start on web application test automation with Robot Framework. I planned to write about how to actually write some test cases but didn’t have the time.

It’s also good to remember that “If you can not measure it, you can not improve it.” (Lord Kelvin) so I covered it with article about monitoring Java EE application with JavaMelody. JavaMelody proved to be very handy tool to do monitoring in different levels and was quite easy to set up. It can show information about e.g. CPU and memory, HTTP requests, SQL queries, Spring beans and REST calls. And it is said that the overhead is so low that it can be enabled continuously even in production environments.

I’m quite satisfied about the articles I managed to make and especially about Robot Framework and JavaMelody which have been helpful on many occasions. During 2013 I also moved the blog to new VPS provider and although there was also an idea to make a new custom theme, it didn’t materialize. Maybe next year.

Planning ahead

For the coming year 2014 the plan, as usual, is to write approximately one article per month. Blog’s draft folder has some items about JavaScript and charts, Munin plugins and general software development and some sysadmin posts but it will be seen if they mature enough for full posts. I will maybe also write something about Sailfish OS development and Sailimgur.

So subscribe to the RSS feed and stay tuned. And if you can read Finnish check also my other blog. Will the new year be better than the previous? Who knows, but at least you can try your best to make that so.

Optical drives are nowadays less common on laptops which makes making bootable install medias a little easier. Yes, easier by requiring you to use USB thumb drives. There are different ways to achieve what you want and here is couple of examples how to transfer Linux install media to USB thumb drive on Windows and Mac OS X.

First you need to download the install media of your choice as ISO image which you want to transfer to your USB thumb drive. In theory you need a media which is meant for USB thumb drive or is a Hybrid ISO image but practically all modern distributions’ media are Hybrid ISO images.

Windows

Select Diskimage and click the “. . .” button to find and open the ISO file. Make sure Type is USB drive and the Drive letter is what your Windows detect and click OK. Your ISO content will be copied to USB thumb drive.

Next you also need to copy the ISO file that you used with UNetbootin to the root of the USB thumb drive, e.g. the ISO file should be F:\linuxmint-16-cinnamon-dvd-64bit.iso

Now you can proceed to boot up the machine that you want to install with e.g. Linux Mint 16.

Linux

Creating bootable USB thumb drive in Linux is much simpler. You just need install media which is meant for USB thumb drive or is a Hybrid ISO image. With hybrid images, you can simply write the content of the ISO and make a bootable USB stick which can install Linux by using the ‘dd’ command.

You just need to have the ISO image and one command:

$ sudo dd bs=4M if=linux-image.iso of=/dev/sdX oflag=direct

$ sudo dd bs=4M if=linux-image.iso of=/dev/sdX oflag=direct

Where sdX is the device where your USB thumb drive is, e.g. sdb. If your system doesn’t support ‘oflag=direct’, you can just leave it out as it is simply intended to speed up the process a bit. If you leave the bs-option off the default will be 512 bytes and takes forever.

Mac OS X

OS X’s DiskUtil can’t burn ISO images to USB and gives “Invalid argument” but fortunately just like with Linux also Mac OS X makes things easy for you. You just need the install media which is meant for USB thumb drive or is a Hybrid ISO image. With hybrid images, you can simply write the content of the ISO and make a bootable USB stick which can install Linux by using the ‘dd’ command.

$ dd if=linux-image.iso of=/dev/rdiskX bs=8192

$ dd if=linux-image.iso of=/dev/rdiskX bs=8192

Where rdiskX is the device where your USB thumb drive is, e.g. rdisk1. You can see the device (disk identifier) from DiskUtility.app by selecting your USB stick and checking the Device section.

It’s useful to notice that on OS X you can use the ‘/dev/rdiskX’ device which stands for raw disk and is much faster than the block level device ‘/dev/diskX’ which is buffered and where data undergoes extra processing. For example rdiskX gives “823132160 bytes transferred in 625.470137 secs (1316022 bytes/sec)” and diskX gives “823132160 bytes transferred in 1296.675998 secs (634802 bytes/sec)”.

If you get an error e.g. “dd: /dev/rdisk2: Resource busy” then you have to make sure the Mac OS X isn’t using it and thus unmount it’s partitions. Open the DiskUtility.app, and on your USB hard drive, unmount any of it’s partitions. Do not eject the USB hard drive.

Munich in Autumn isn’t maybe the most predictable travel target but for many decision-makers, experts, project managers, IT architects, consultants and users it’s the place to go when Fujitsu’s largest IT-event in Europe took place on 6th and 7th November. I was one of the invited bloggers to visit Fujitsu Forum 2013 and again it was very insightful and exciting two days. With the motto “Reshaping ICT, Reshaping Business and Society” there were over 11,000 attendees around the world getting new information and looking for strategic and operative options for modernising and developing their IT.

Fujitsu Forum 2013

Fujitsu Forum 2013 in Munich is Fujitsu’s largest IT-event in Europe and was this year attended by more than 11,000 IT experts from around the world. With keynotes, breakout sessions, expert talks, exhibition and evening event the Fujitsu Forum is good place to get new information and looki for strategic and operative options for modernising and developing IT. Just like last year, I was invited by Fujitsu LIFEBOOK4Life campaign with three other insiders aka bloggers to spend couple of days at the Forum and to get insight what the future of IT might look like.

This year the Forum was held with the motto “Reshaping ICT, Reshaping Business and Society” and the focus was more about how the technology affects us and how we can make it work for us. Or in business words: “How to combine business processes and IT to form a stable basis that will ensure growth and success in the future.” The main topic was reworked from last years’ “Reshaping ICT – Reshaping Business” and the ICT trend could be easily seen from the buzzwords I heard also this year. Consumerization, BYOD, tablets, virtualization, in-sync, cloud and “One workplace – on any device” were still hot topics but maybe not so strong than last year as most of the hype has passed.

Fujitsu Forum 2013 startsL4L team and insiders at press lounge

Sessions

Our great LIFEBOOK4LIFE team had planned two keynotes and one breakout session which meant the days wouldn’t be so packed and we would have time to share our insights and to stroll at the Exhibition area.

We started with Defining Your Workplace Strategy: Our Recommendation to IT breakout session by Meinolf Althaus (Fujitsu) who told us how to manage the different users’ needs for the workplace. The three points were: Analyze your users and create user profiles (one size does not fit all); Accommodate multiple devices, operating systems, delivery technologies; Rely on a partner which offers an end-to-end portfolio.

Last year we missed the opening keynote but this year we were wiser. The opening keynote provided a view to Fujitsu’s technology and services vision: human centric intelligent society. The keynote presented the idea behind the “human centric intelligent society” vision and also the exhibition was also built on the three dimensions presented on the keynote: Technology and infrastructure and integration (optimize ict systems from end to end); process and information (power business and society with information); people and innovation (create innovation through people). The main point was that we are in a global village and virtually everything and everybody are connected in one extensive network. But with global village come some challenges both in business and society sense. For example social media is more pervasive and the volume of information is astounding. The question is, how do you mine, analyze and act upon that data? For more detailed information check Marco Rossi’s short recap of the session and how Fujitsu’s services address this topic.

Our last session was on Thursday and I have always liked Dr. Reger’s presentations which are both informative, clever and witty. The CIO Session: “All things considered” – How the internet of things changes our world gave us good ideas what are the technology trends that are going to affect our lives and businesses in the next few years and how the world could look like when every possible machine has it’s own IP address.

Internet of things = there is no people and cat has IP address – Dr. Joseph Reger

In short “All things considered human centric intelligent society is the internet done right. The job is done by the things for the people. Internet of things will make our lives more complex but with multiple opportunities.”. The road to Internet of things isn’t yet paved and there are still many problems to solve like starting with IPv6 becoming mainstream, standards to control the devices and security issues. Reger gave some Sci-Fi like examples what kind of possibilities the Internet of things will present to help our everyday life for example by sensing different things and acting based on that information. It’s worth to check out the video recording of the session from the Documentation or read the written recap by Fujitsu’s Marco Rossi.

Virtually everything and everybody are connectedEverything will have IP address, like light bulb

Overall, I have to admit I didn’t get a good grasp about the message of the Fujitsu Forum 2013 which had a bit different aspect to cover than last year. The same ideas were on the background but from my Fujitsu Forum 2012 insights the message is easier to pinpoint: “The work and workplace is changing to support mobility and working anywhere anytime. The borders between business and private life are blurring, consumerization (same devices home and work) and BYOD (bring your own device) are coming more common. We should think more about user-oriented workplace. In the future the devices we use will be more diverse and we have more of them. Thus there will be need for virtualization, syncing data and support for “One workplace – on any device” ideology. Different tasks have different needs for the device so we need a dynamic desktop experience.” Maybe the topic felt this year more vague as I attended two keynotes which had more abstract than concrete ideas how the workplace might look like compared to last years’ three breakout sessions.

If you weren’t able to attend the Forum or some particular session you can re-experience the Fujitsu Forum 2013 and watch or download the presentations from Fujitsu Forum 2013 documentation page.

Exhibition

Between the breakout sessions and keynotes we had time to check out the exhibition are with 250 highlights from the portfolio offered by Fujitsu and its partners. I especially liked innovative ideas like The Office 21: Intelligent Desk which showed how to make the physical desk better and the unified communication & collaboration setup (article by FujitsuFans.com) which presented how video conferencing and sharing ideas can work more seamlessly. I also didn’t know that Fujitsu also makes ATMs.

And of course as a technology geek getting my hands on to Fujitsu’s new LIFEBOOKs Ultrabooks with IGZO display was nice. Both the versatile T904 convertible ultrabook/tablet and the ultra-slim U904 were beautiful devices and I especially liked how the RJ-45 port was designed. Also the semi-rugged STYLISTIC Q704 tablet had nice use cases and accessories to support it.

For more detailed information about the different topics presented at the Exhibition check the Fujitsu Fans -blog with the forum tag and the YouTube playlist. I just don’t understand how Abassin who keeps the Fujitsu Fans blog (MixBlogeu in YouTube) had time to do so many hands-ons and interviews at the Forum :) If you’re interested to know more about the new LIFEBOOK Ultrabooks U904 and T904 check the hands-on articles from the Fujitsu Fan blog as they present nicely what kind of Ultrabooks they are.

Networking and entertainment

And lest the whole two days be just about technical topics there was an Oktoberfest themed evening event on Wednesday. This year the event provided something different and was more social than previous years’ concerts with Anastacia (2011) and Amy McDonald (2012) which were also nice. At the “Oktoberfest” there were special Bavarian dishes, such as crispy roast chicken (Hendl), spiced strips of pork belly (Hüttenspeck) and of course pretzels (Obatzda und Brezn) and excellent beer. And it was not just food and drinks as there were playful games like sawing a log, throwing a pint, hammering nails and throwing balls. If the games weren’t for you, you could take a ride in a carousel, ferris wheel or swing (schauke).

Summary

Fujitsu Forum 2013 is over but the new insights are just starting to form up to new ideas how to do things better. This was my third visit to the Forum and every year has been interesting and a bit different and great on their own kind of way. This year the sessions, keynotes, I listened were more about ideas than doing and they gave a good view how the future might look like with trends like Internet of things and how Fujitsu sees the future and business through their “human centric intelligent society” vision. I also got to see great looking Ultrabooks, interesting new concepts and products and I really liked the Oktoberfest themed evening event which was more social than previous years’ concerts.

The three days in Munich went fast and it was again great to visit Fujitsu Forum and to see other Insiders and the campaign crew. Thanks LIFEBOOK4Life and Fujitsu.

In two weeks time it’s again that time of the year when business- and IT decision-makers, experts and specialists, project managers and IT architects, consultants and users travel to Germany to visit Fujitsu’s largest IT-event in Europe: Fujitsu Forum 2013. And that’s also the destination where I’m going as I’m one of the bloggers invited by Fujitsu’s Lifebook4Life campaign to visit the forum. This year the event is held 6th and 7th November at the ICM in Munich and there will probably be over 12,000 attendees around the world getting insights and looking for strategic and operative options for modernizing and developing their IT.

The motto this year, “Reshaping ICT, Reshaping Business and Society”, highlights the key theme that is addressed by all aspects of the event: How to combine business processes and IT to form a stable basis that will ensure growth and success in the future. So what can you expect from Fujitsu Forum 2013? The event overview gives you some idea about the different topics covered but let’s look which events I’m looking forward to the most.

The presentations I picked from Keynotes and the list of 52 Breakout Sessions for my Fujitsu Forum 2013 agenda span all of the three conference tracks, “Create innovation through people”, “Power Business and Society with information” and “Optimize ICT systems from end to end”, and cover topics like cloud services, creating value from enabling innovative IT models with mobile technologies and information security and data protection. Overall I selected 10 sessions to my agenda but pruned it to one Keynote and four Breakouts as you don’t have time for every interesting topic.

Adopting new computing devices and delivering concepts to provide optimized workplaces for the different user roles within a company present a new challenge. An optimized end-to-end selection, from back-end to workplace devices and a suitable combination of desktop delivery methods is essential to guarantee high availability, flexibility and user satisfaction. This session looks at the various ways to identify the ideal workplace strategy.

Leveraging ICT to generate New Value for society is essential to enable sustainable growth and advancement of societies. This session highlights how Fujitsu’s R&D is being leveraged toward realizing a Human Centric Intelligent Society envisioned by Fujitsu, by supporting the Fujitsu Technology and Service Vision, to be achieved through Three Actions: Create Innovation through People’s Activities; Power Business and Society with Information; and Optimize ICT Systems from End to End. We introduce how it will contribute to societal growth and realization of a prosperous society in which people can thrive.

There are fundamental changes taking place in society, new technology and changing demands are redesigning the workplace. The enterprise workplace will dramatically improve interactions across applications and between workers, driving productivity and improvements in work quality. This session will show you how new technologies support you at your workplace and how to ensure efficiency in and outside the office.

Healthcare costs are rising and the burden of managing long term conditions threatens to destabilize whole economies. Clinical information has to flow smoothly to ensure a seamless patient journey through the healthcare system and the safe practice of medicine has special requirements in terms of the data value chain. The right information, in the right place, at the right time. This session will give examples of how IT can power healthcare in new and exciting ways and underpin the radically different processes we need to adopt, if we are to avoid meltdown in our healthcare systems.

And lest the whole two days be just about technical topics there’s the Oktoberfest themed Evening Event on Wednesday. Excellent entertainment and networking opportunity with drinks together with special Bavarian dishes, such as crispy roast chicken (Hendl), spiced strips of pork belly (Hüttenspeck), bread and dripping (Schmalzbrot), chive bread, savory cheese spread with pretzels (Obatzda und Brezn). Hmmm, and now I’m hungry :)

There are times when you have to do a clean install of your Windows 8 but if you have just an upgrade key you need to make couple of extra hoops before you can activate the new install. The upgrade key doesn’t prevent you installing to a clean disk but when you try to activate, you get an error 0x8007007B, saying your product key can only be used for upgrading. Another fine example how Microsoft makes things complicated for legitimate users.

Navigate to “HKEY_LOCAL_MACHINE/Software/Microsoft/Windows/CurrentVersion/Setup/OOBE/” and change the MediabootInstall key’s value from 1 to 0.

Open the Command Prompt (Win + R, type cmd). Right-click on the Command Prompt icon and run it as an administrator.

Type slmgr -rearm and press Enter.

Reboot Windows.

After that is done and you get back into Windows, you should be able to run the Activation utility and activate Windows as normal, without getting an error.

Apparently you can also call Microsoft Support and they will walk you through the proper way doing this because they understand you may have bought a new drive. There is a dialog you can get to in system tools where it will ask you for a numerical code. The support personnel will give you the number, you click “OK” after typing it in, then go the activation again and it works. This process is likely doing the work around mention above, but through an approved administrative process.

By the way, restarting Windows 8 is most easily done by left-clicking once on an empty spot on the desktop and holding Alt + F4).

Hilighting code blocks in Worpdress blog posts can be achieved with different plugins and one of them is the WP-Syntax plugin which uses GeSHi source code highlighting and supports a wide range of popular languages, highlighting with or without line numbers and maintains formatting while copying snippets of code from the browser. The plugin highlights code using HTML’s <pre> tag, e.g. <pre lang=”java” line=”1″> for Java and line numbers starting at 1.

Source code highlighting makes it easier to read examples but sometimes if your layout isn’t 900+ pixels wide, you may end up with horizontal scrollbars for wider code blocks as you can’t always make it 80 characters. But fortunately you can make the plugin auto-expand the code block when hovered over. You just have to edit the plugins code a bit as Derek MacDonald explains.

For some reason the code to auto-expand the block when hovered over has been commented out in the plugin and to enable it you’ll have to edit three files:

wp-syntax/wp-syntax.php
In method enqueueScripts(), uncomment the line that invokes wp_enqueue_script() so wp-syntax.js will be loaded.

wp-syntax/css/wp-syntax.css
Ensure the overflow, overflow-x, and overflow-y properties of class .wp_syntax are set to hidden.

wp-syntax/js/wp-syntax.js
It might be better to remove the box-shadow CSS property added during the mouseover event because it doesn’t look good in most designs.

The mouseover/mouseout expand/collapse events will be immediate with no delay.

Running Apache 2 and PHP is simple with mod_php but there are more efficient alternatives like using PHP-FPM (FastCGI Process Manager) which is an alternative PHP FastCGI implementation. With it the PHP process runs standalone without the need for a web server and listens for incoming requests on either a TCP or a Unix socket. Web servers can connect the PHP process and send requests using the FastCGI protocol. It solves mod_php’s problem of spinning up and destroying PHP instances with every request and thus is more memory efficient and provides better performance.

These instructions are for CentOS 6.4 but the process should however work similarly with other Linux distributions.

Setting up the PHP-FPM

Install the FPM-CGI binary for PHP and add it to start after server reboot:

# yum install php-fpm
# chkconfig --levels 235 php-fpm on

# yum install php-fpm
# chkconfig --levels 235 php-fpm on

Configure the PHP-FPM pool in /etc/php-fpm.d/www.conf to use sockets and enable some status information for e.g. Munit:

Setting up Apache and mod_fastcgi

Apache can be configured to run FastCGI with two modules: mod_fastcgi and mod_fcgid. The difference is explained at Debian bug report #504132: “mod_fcgid passes just one request to the FCGI server at a time while mod_fastcgi passes several requests at once, the latter is usually better for PHP, as PHP can manage several request using several threads and opcode caches like APC usually work only with threads and not with processes. This means that using mod_fcgid you end up having many PHP processes which all have their very own opcode cache.”

In short: mod_fastcgi is better.

Install mod_fastcgi

So we need to get mod_fastcgi which isn’t at the time found from CentOS default or EPEL repos but from RPMForge or by building it from sources.

Configure mod_fastcgi

If you have php enabled disable it

# mv /etc/httpd/conf.d/{php.conf,php.conf.disable}

# mv /etc/httpd/conf.d/{php.conf,php.conf.disable}

Set up a (non-existent) directory that Apache can route the requests through. That directory must be available to Apache and it might be /usr/lib/cgi-bin/ so the routed file is then e.g. /usr/lib/cgi-bin/php5-fcgi.

We add handler and action which sends all requests of PHP to the virtual URL created above, which is in turn then sent to the external FastCGI server. We also add configuration to have some status information about our PHP-FPM.

Writing documentation or user guide isn’t the most interesting part of the software project but it’s still needed to be done and important for the end users. There are many ways to achieve documentation but using Markdown is good way to make it easier. After that you just need to convert it to HTML and as I couldn’t find a suitable Maven plugin to do that nicely I wrote one myself: markdown-page-generator-plugin.

Software developers are a bit lazy to write good and up to date documentation and as I needed to write the API documentation for the web application as separate HTML pages I got to know the issue first hand. I like writing documents but typing the API documentation as static HTML wasn’t very effective or fun. Fortunately it was quite easy to do it with Markdown which allows you to write documentation using an easy-to-read, easy-to-write plain text format which you can then convert to HTML. After that the only problem was to enable easy HTML generation directly from Maven Lifecycle and thus was the “Markdown to HTML Page Generator Maven Plugin” developed.

Markdown to HTML Page Generator Maven Plugin

“Markdown to HTML Page Generator” is a simple Maven Plugin which reads the input directory for Markdown files with .md suffix and outputs HTML files to target directory. There are couple of configuration options to set the input/output directories and if you want to add header and footer files to make it structurally valid HTML. The plugin uses pegdown Markdown processor.

Source code

Usage

Using the plugin could be easier but as it isn’t in Maven repository you have to e.g. add it as Maven project to same Eclipse workspace as your project and build it. After that add it as a plugin to your build and it’s run automatically when processing sources. You can also run it from command line with “mvn com.ruleoftech:markdown-page-generator-plugin:generate”.

Setting up LAMP stack for web development on OS X can be done with 3rd party software like MAMP but as Mac OS X comes with pre-installed Apache and PHP it’s easy to use the native setup. You just need to configure Apache, PHP and install MySQL.

Setup Apache2

Set up the Server Name to localhost to suppress the warning about fully qualified domain name and enable PHP module.

As we are using the Homebrew package for MySQL and the default php.ini file then PHP is trying to connect to MySQL through the default_socket at /var/mysql/mysql.sock which doesn’t exist as MySQL is using /tmp/mysql.sock. Just change all instances of /var/mysql/mysql.sock to /tmp/mysql.sock.

Software development is much more than just coding application by requirements and deploying it to production as the real work really starts after it has been shipped: maintenance, improvements and problem solving. And for that it’s good to have some data. It’s said “if you can’t measure it, you can’t improve it” and in ideal situations you plan for measuring your app but often you have to do it in production to detect and diagnose performance problems. There are different ways to measure a Java EE application and one good tool to achieve performance and monitoring statistics is Javamelody which has low overhead, is non-intrusive, informative and simple to setup but still very capable.

“If you can not measure it, you can not improve it.” – Lord Kelvin

Application performance can be measured by two main methods: computational resources used by the application and the performance as seen by a user of the application. By measuring these quantities we get an empirical performance baseline of the application which then can used to detect changes in performance. By using performance monitoring, which is an act of non-intrusively collect or observe performance data from running application, we get measurements to identify or isolate potential issues from real operation of an application without having a severe impact on runtime responsiveness or throughput.

There are different ways to achieve performance and monitoring statistics and it’s useful to plan for measuring application’s performance while still in development as later on you surely want to know how it is performing. You can do it by leveraging different JMX-based tools like Metrics and Servo but if you didn’t really plan for it you still have choices. You can use tools like standard JDK tools (jconsole, jstat, jmap, jstack, hprof) for low level JVM monitoring or some monitoring application like Javamelody. And as time is always scarce in development it’s useful to deploy a tool like JavaMelody which provide quick and easy access to performance monitoring. But if you are interested of some narrow focus measurement you’re better of with some custom tool.

Monitoring with JavaMelody

JavaMelody is an open source (LGPL) application to monitor Java or Java EE application servers in QA and production environments. It is a tool to measure and calculate statistics on real operation of an application depending on the usage of the application by users and is mainly based on statistics of requests and on evolution charts which can be viewed on the current day, week, month, year or custom period. The statistics can be viewed on a HTML page and sent as PDF reports by email.

JavaMelody lists the following use cases:

It allows to improve applications in QA and production

Give facts about the average response times and number of executions

Make decisions when trends are bad, before problems become too serious

Optimize based on the more limiting response times

Find the root causes of response times

Verify the real improvement after optimization

Setting up

Setting up JavaMelody is quite easy and the needed steps are covered in User guide. Integrating JavaMelody to your application can be done in less than 10 minutes, by automatic discovery of environment: it only requires to copy 2 jar files and to add 10 lines in a xml file. But for more detailed and wider measurements you need couple more lines to xml files and here are the changes I made for our Wicket, Spring, JPA, Hibernate -Java EE project.

JavaMelody and dependencies

Getting JavaMelody and it’s dependencies is easy with Maven2 and you just need to add javamelody-core and if you want to have PDF reports also iText in your pom.xml:

JavaMelody needs just a monitoring filter before the description of webapp’s servlet in WEB-INF/web.xml or not even that as described in the User guide. But usually you want a little bit more than the minimum 10 additional lines in web.xml.

By adding the customResourceFilter you get customized look & feel, you can exclude some urls from statistics with url-exclude-pattern regular expression pattern, eamil-parameters are for weekly reports and by including monitoring-spring.xml you can monitor DataSource with a Spring post-processor and objects initialized with Spring.

For restricting the access to monitoring statistics I added the following to the Spring Security’s applicationContext-security.xml. There are also other options for security. The key point here is to make sure, that the monitoring-filter in web.xml (above) is defined after the Spring Security filter chain.

If the application to monitor contains some objects initialized by Spring, EJB or Guice their methods execution can also be added to statistics. As described in the User guide for monitoring Spring Business facades there are couple of options and for example with JdkRegexpMethodPointcut in applicationContext-web.xml you can catch objects with regular expression like “all that have Service in their names”.

JavaMelody already shows datasource and SQL information but it’s also possible to display information and statistics on the database like the longest requests in cumulative time with display of the cpu time and of the elementary cost. For showing that information the user in the database used by the monitored application must have the necessary rights to read those information and statistics.

In Oracle database the request practically means select * from v$session and you can grant access as ‘system’ user with GRANT SELECT any dictionary TO myapplicationuser. Strangely granting just the select for v$session (GRANT SELECT ON sys.v_$session TO myapplicationuser;) wasn’t enough.

Reports

There is also a possibility to get weekly, daily or monthly report in pdf format sent by email to one or several people. It needs iText library for webapp and Java’s JavaMail and Activation libraries in your server for the mail session.

With WebLogic you add some email parameters in webapp’s web.xml (above) and configure a Mail Session in WebLogic AdminServer: Services > Mail Sessions with JNDI Name and JavaMail properties. The report provides the same information you can find in monitoring web page with high and detailed level information.

Statistics’ storage

Javamelody data is stored in files on disk and there are 2 types of files: *.rrd files for values in graphics in RRD format (using jrobin library) and *.ser.gz for values in statistics.

The performance statistics are stored to temp/javamelody directory by default which means /tmp/javamelody in Linux and in Windows when running webapp in Eclipse something like Users/developer/AppData/Local/Temp/javamelody. If you want to reset the counters just delete all the files. The location can be changed with storage-directory parameter and if the name of the directory starts with ‘/’, it is considered as an absolute path, otherwise it is considered as relative to the temporary directory.

Viewing performance monitoring

After you have set up your webapp to have the monitoring you can see statistics at URL like http:////monitoring depending your configuration.

Some example JavaMelody statistics from development:

Graphs about CPU, memory, etc.

HTTP requests

SQL information

Detailed SQL information

Spring beans

Other information like system, threads, caches

Overhead

Monitoring and filtering doesn’t come without costs and there has been discussions on JavaMelody wiki about the overhead the monitoring does. It is said that the overhead is so low that it can be enabled continuously in Quality Assuarance environment and if no problem arises in QA, also continuously in production environment. And with little overhead you will be able to know what needs optimizing in the QA or production servers so that the overhead of JavaMelody will practically be negative.

The discussion contained some notes:

Architecture of JavaMelody is lightweight so it has a lower overhead as compared it to other available solutions

It is only statistics and not events so the overhead of memory is quite minimal.

It does monitoring not profiling: there is no instrumentation of classes and instead “interceptors” for http, jdbc, spring or ejb3.

No database and no recording of each events even in a file or over the wire: only statistics of requests are kept. The overhead of cpu is minimal with no I/O on the wire and minimal I/O on disk (just to take a backup of statistics at a regular interval). The overhead of some other good monitoring solutions is in the recording of each event in a database or in a master server.

You have the choice to use centralized collect server which unloads the memory, the backup storage and the generation of reports to another server while adding I/O on the wire for sending deltas of the statistics.

Summary

“If you can not measure it, you can not improve it.” It doesn’t matter how you do it but if you want to use real user data and environment just do it non-intrusively with performance monitoring tools like Javamelody.