Michael's BIRT Blog

In December, Actuate released the latest generation of the BIRT iHub (iHub3) for distribution. The completely updated iHub 3 simplifies the design and deployment of personalized analytics and insights via a single platform that integrates BIRT based, visually appealing, interactive application services, predictive analytics services, and customer content services. This release promotes increased creativity, efficiency, and productivity for BIRT developers creating internal and customer-facing applications that can deliver personalized visualizations and analytics securely to millions of end users, on any device, at any time.

The BIRT iHub 3 release features the following:

Enhanced User Experience via Data Visualizations and Presentation

HTML 5 based data visualizations that support any data source and any device

Appealing visualizations by default

New Android phone app for consuming BIRT content

Rich, modern interface that provides a new, simplified, and consistent user experience

Enhanced Developer Productivity and Broader Data Access

Faster time to market via reduced time to develop and deploy applications

New BIRT Data Model that simplifies BIRT meta data management

Access to more Big Data sources and Excel data

Mac OS and Windows 64 bit developer tools

Enable easy migration from open source BIRT to commercial features

Improved IT Efficiency via a Unified Deployment Platform

Centralized system management

Quick installation and configuration

New built-in monitoring and utilization system

Simplified management for all styles of deployment including SaaS or on-premise

Over the next several weeks, I’ll be highlighting these areas in a blog series. Download the trial version of the iHub 3 to explore the new features and look and feel yourself. If you have questions or comments, feel free to post them in the comments section below or in the iHub forum.

For those that don't know, Cloudera Enterprise 5 is one of the industry-leading Hadoop data management platforms. It provides a single integrated platform for bringing diverse users and application workloads to a common infrastructure with security, enterprise-grade data auditing, fault tolerance, automated data backup, system and data management, and more.

With BIRT Designer Professional and BIRT iHub now certified with Cloudera 5, it becomes easy to rapidly gather, filter, and analyze massive amounts of data. This allows for critical insights to be formulated and quickly communicated with end users through meaningful and interactive BIRT-based visualizations. All you have to do is connect to your Hadoop data via HiveServer2 or Cloudera Impala.

Loading Your Data

Data from HDFS can be loaded into your Cloudera HiveServer2 in several different ways. For running HQL queries through the command terminal, you have by simply typing "beeline" into the terminal and then connecting with your database url, as seen in the image below:

Or you can use the Hue interface within Cloudera Manager to manage HDFS and run Hive and Impala queries.

Connecting to Your Data

Now that you have data loaded into your HiveServer2, you can either write queries against that data from the terminal or Hue, as seen in the image above. Or, you can use a tool like BIRT to grab the data via a JDBC connection and turn it into something meaningful for your end users.

BIRT Designer Professional has a Cloudera specific data source. With open source BIRT, you'd use the JDBC data source. In this blog, I'm using BIRT Designer Professional.

First things first, grab the 0.12.0 Hive2 jars from your Cloudera install and add them to your BIRT install in the folder <BDPro location>/eclipse/plugins/org.eclipse.birt.report.data.oda.jdbc_4.2.3.v20131216-0430/drivers/. In the Cloudera VM I used, these were located at /opt/cloudera/parcels/CDH/lib/hive/lib/.

Now, all we do is create a new data source in BIRT and choose the Cloudera Hive Data Source type.

Next, you'll enter your connection information and test to make sure you're able to connect to your server. In the below image, a connection to the HiveServer2 is made. If you were wanting to connect via Impala, you'd use a URL like: jdbc:hive2://192.168.40.130:21050/;auth=noSasl

With your connection made, you can now create a data set using this data source and write your HQL queries.

What Now?

Now that you're able to connect to your data, it's just BIRT. Just as with any other data, you can do further computations and joins, create tables, charts, crosstabs, etc. to display your data in the way that is useful to your end user. Then, you deploy the reports to make them accessible to your users.

For the certification, I took these reports and deployed them to my BIRT iHub3 and scheduled 20 reports to run against HiveServer2 and Impala performing various HQL query functions. For those that don't know about iHub, it's a very powerful BIRT based platform that provides security, scheduling, distribution, interactivity, and more. To learn more about iHub, see the product page. You can download a free trial here to experience all that Actuate and BIRT can do for you and your Cloudera-managed Hadoop data.

If you have questions about using BIRT with Cloudera, feel free to post questions/comments in the blog comment section or ask questions in the community forums. Thanks for reading.

This past week at EclipseCon 2014, I had a session about creating data driven web applications with BIRT. This session was meant to walk you through the entire process of design and deployment of a BIRT web application. I started with an introduction to BIRT, including a high level architecture overview. This was followed by a walk-through tour of the designer and a look at the many deployment options available, including the Sample Viewer, Report Engine, Report Engine with JSF, BIRT Viewer Toolkit, and the iHub F-Type. If you were able to attend and were wanting a copy of the slides or missed it and would like to see what it was all about, a link to a PDF version can be found below. If you have any questions, feel free to post in the comments, ask a question in the forums, or private message me.

If you've logged into BIRT Exchange during the past few months, you've surely noticed some changes! There has been a complete structure and domain change. BIRT Exchange is now the Community portion of the new BIRT Developer Center. The goal of the new developer center is to make it easier for developers at all stages of knowledge of BIRT. Whether you're new to BIRT and just wanting to know what all BIRT does, or if you're a BIRT guru and want to show others the cool designs you've created or help support the community, there's a place for you. We'll also be rolling out more features and areas of the site in the near future, making for an even better user experience. We're very excited about the direction of the new site and hope you are too. Feel free to contact us and let us know what you think.

I'm pleased to announce the members of the BIRT Developer Center Community Leaderboard for October, November, and December 2013. The purpose of this Leaderboard is to honor the top contributors to the community each month and to have a little fun. Scores are also posted on the Developer Center Community. You can recognize past and current Leaderboard members by special badges on their forum and DevShare posts:

To determine the leaders, each contribution made by a Developer Center Community member is assigned a weighted point score that roughly corresponds to its effort/value. For example, starting a forum topic by asking a question will score you 1 point and answering someone's question scores 3 points. Members with the top summed scores each month are declared the leaders for that month and posted to the Leaderboard. Over time we evolve the scoring system as needed for fairness and value to the community. We even give points for rating forum posts.

So please contribute to the Developer Center Community. It's not difficult to get a decent score and we'd love to see you on the Leaderboard next month!

You can find details on the Community Leaderboard page. If you have any suggestions or comments, please add them to this blog post (they're worth 2 points).

External JavaScript libraries have always been able to be used with BIRT. BIRT 4.3 has made including these types of libraries easier with head.js. If you're using a version prior to BIRT 4.3, I'll also cover a way to be able to include head.js yourself. In this blog I'll be showing the use of jQuery and jVectorMaps to show state GDP data in a map.

First, I'll start with how this can be done in BIRT 4.3. A new script area has been added called "clientScripts". This script area can be found by clicking in an empty spot on the report, going to the script tab, and then choosing "clientScripts" from the drop-down.

Here, you can use head.js to load the necessary JavaScript libraries with a call like:

head.js(jsFile1,jsFile2,...,callback function);

Because of the timing of when files are loaded and when the report body is available, the callback function is probably only going to be used for testing. For example, you could write an alert statement that is called after your JavaScript libraries are loaded. You must also make sure that none of the libraries you're using conflict with any others. In our example, jQuery does conflict with other libraries used within the viewer, so we won't use the "clientScripts" area to load our libraries because jQuery must be the first one loaded.The next step that you'd normally take would be to use the head.ready() call in a HTML text control to execute the script that uses the JavaScript libraries you loaded in your head.js() call. That looks like this:

head.ready(function (){//your script using your JS library});

To see an example of how this is used without conflict and to see the other features added in 4.3, take a look at this blog.

In our example, where we have a conflict with a library that's used in building the viewer, we call head.js() from the text box. Of course, even if you don't have conflicts, you can avoid the "clientScripts" script area and call your head.js() here, as well. Since we're calling head.js() from the report body, we can use the callback part of the function to run the script we're wanting to run. Here is what our head.js() call will look like in our text box:

This code takes the data from our dataSet and creates a map to shade each state based on the value. A style sheet is also added to the report to help with the look and feel of the map and labels. You can see how this is added in the styles section of your Outline view. Once we've done all of this, when we run the report, we get this as our final output:

If you're using a BIRT version prior to 4.3, you can still do this by using the method in this devShare post. With this method, you can also skip the Variables and simply use the head.js() call in your HTML text box like we did above.

The example shown in this blog post can be found in the devShare. Feel free to comment here or there if you have questions or issues.

BIRT 4.3.0 was released a couple weeks ago with the release of Eclipse Kepler. This blog will talk about the new features that are available in this new release. Below is the list of features that will be discussed. Any features that were back-ported to BIRT 4.2.2 are noted with an asterisk (*).

Feature List

New Microsoft Office Emitters

New POJO Data Source

* New Cassandra Scripted Data Source

New MongoDB Data Source

Easy Client-Side JavaScript Library Inclusion

New Microsoft Office Emitters

BIRT currently supports emitting report output in many formats. These include HTML, paginated HTML, DOC, XLS, Postscript, PPT, PDF, ODP, ODS, and ODT formats. BIRT is designed to be extensible and provides extension points for many components within the designer and runtime. The emitter framework provides an extension point to allow developers the ability to emit essentially any desired output. In this release BIRT will offer enhancements to all three of the Microsoft Office emitters.

For PowerPoint and Word, support for Office 2010 has been added with the inclusion of emitters for DOCX and PPTX formats. For Excel, the popular SpudSoft Excel Emitters were added to BIRT which includes emitters for both XLS, named XLS_SPUDSOFT as the standard XLS emitter will remain, and XLSX. The new XLS & XLSX emitters are able to output pictures, which means charts can now be seen in your Excel export. These emitters will also give a more Excel-like output rather than trying to make the Excel output look like the HTML, meaning no merging of cells. Multiple sheets are also an option with the new XLS & XLSX emitters.

All of the above export formats are available within the designer, in the viewer, or by using the Report Engine API.

Export Options

PPTX

DOCX

XLS/XLSX

NEW POJO Data Source

As with all information applications, the visualizations are only as good as the data that is analyzed and retrieved. BIRT is no exception and provides extensive support for retrieving data. BIRT currently supports drivers for JDBC, XML, Web Services, Flat Files, and Hive. In addition, BIRT also supplies a Scripted Data source that can be implemented in Java or JavaScript that can be used to access virtually any data source that is structured or contains an API. In the past, when you wanted to connect to a POJO (plain old Java object), the scripted data source is the one you would use. In this release BIRT has added support for POJOs to be used as a data source making using this type of source easier than ever.

Next, we create a new data set using our POJO data source and choose our CustomerDataset.java class.

Then, select the column mapping class used to define the data set columns and hit next.

Select the columns that you want to include in your data set.

Finally, hit finish and preview your results.

*New Cassandra Scripted Data Source

BIRT 4.2.2 and later supports scripted data source connections for use with Cassandra distributions, including:

Apache Cassandra

Datastax Cassandra Community

Datastax Cassandra Enterprise

BIRT connects to and queries a Cassandra data source using the Hector API. You must copy the Hector client library JAR files to the appropriate folders for both BIRT Designer and the BIRT runtime deployment application. You can download the JAR files from here.

For the designer, copy the JARs into the /plugins/org.eclipse.birt.me.prettyprint.hector_version/lib/ directory. Depending on the runtime type you're using, you'll place the jars in different locations. For OSGi BIRT runtime deployment, copy the JAR files to the /plugins/org.eclipse.birt.me.prettyprint.hector_version/lib/ folder and restart the runtime deployment application. If the "Servlet ViewerServlet is not available" error message appears, copy the org.apache.commons.logging_version.jar (available in WEB-INF/platform/plugins/) to the birt/WEB-INF/lib/ folder. This is a known issue with tomcat-6. For non-OSGi BIRT runtime deployment, copy the JAR files to the /webapps/birt_runtime/scriptlib/ folder and restart the runtime deployment application.

With the Hector jars copied in, you can now create your data source by selecting the Cassandra Scripted Data Source data source type and clicking finish. Then, create your Scripted Data Set by creating a new data set using the scripted data source and then defining your fields. Now that you've created your data set, you can now write your script. To get started with this, select your data set in the Data Explorer and then the Script tab under the design layout window.

Now, you just need to connect to your data using the open script function and then map your data to your rows you defined in the fetch script. To test your data set, open the data set and choose the preview results.

New Mongo DB Data Source

The BIRT project has been designed to facilitate access to as many data sources as possible. Currently BIRT natively supports Flat File (CSV, SSV, PSV and TSV), JDBC Textual, JDBC Graphical Query Builder, XML, and Web Service data sources. This project provides the ability for BIRT developers to access data contained in MongoDB, a popular NoSQL repository.

Now that you have your source set up, you'll use it to create a new data set. In the data set wizard, select the mongoDB data source and hit next. Then, choose the collection you wish to use and select your fields. You can also set up command operations for MapReduce, Aggregate, or Run Database Command.

Once you've selected the fields you want and set up your desired command operations, hit finish and preview your results.

Easy Client-Side JavaScript Library Inclusion

BIRT 4.3 introduces the use of head.js to allow for easy inclusion of JavaScript libraries. To use this feature simply go to the script tab after selecting an empty spot in your report. When here, choose "clientScripts" from the drop down.

Here you'll write a head.js script to load your JavaScript libraries. The head.js call uses the format head.js(file1,file2,...,[callback]); This will look something like the following:

In a HTML text box in your report layout, you'll write your script that uses the JavaScript library you've included. Using head.js's head.ready() function to run your script after all the JavaScript libraries have been loaded will look something like this:

Once completed, you can run your report and see the easy client-side use of your external JavaScript library.

If you have questions on any of these, feel free to post questions in the comments section or in the forums.

I responded to a forum post the other day about having a "tabbed" output so that the user could choose what section of the report they wanted to see by selecting a tab at the top of the report. To do this, you would set up a design with a grid at the top of the design with a single row and a column for each section/table. A HTML text box should be placed in each cell. The text box will contain a title for the section and a link that calls a javascript function that we'll create later. The contents of the text box will look like this:

Each text box is styled to be a tab. The farthest left tab is styled white with borders on the top, left, and right to appear selected. The other three tabs are grayed out with a border all the way around to appear deselected. Once this is done, our tabs will look like this:

Next, you'd create the sections of the report. In this simple example, tables for Customers, Employees, Orders, and Payments from the sample database were created. A bookmark was then added to each table to allow us to access the table in our client side script. If you had more than just a simple table, you could put each section's elements in a grid and bookmark the grid.

Finally, a HTML text box is placed at the bottom of the report with our script. The script takes the section value from the tab link when the user clicks on it. Based on the passed value, the style for the tab is changed to show that the tab has changed. Also, the section display style is set so that the selected section is shown. Here is a look at the full script:

There are only a few days left to get your entry in for the Plug In 2 BIRT Contest - Spring 2013. Be sure to get your entry in by 11:59pm PST on April 30th. If you're still looking to get started, take a look at the contest page. There are some ideas and tutorials there that should help get you started.

Any eligible entry is in the running to win an iPad®. We'll be giving three away, so there are multiple chances to win! Good luck! We look forward to seeing your entry!