Velocity - An Introduction

by Thomas Paul

The Early Years of Servlet Development

I started working on my first servlet application shortly after the Java
Servlet specification was released. It became readily apparent that Servlets
were not designed to separate the Java developer from the HTML developer. In
fact, since the HTML was inside the Java program, even trivial changes to HTML
required the Java programmer to change code, recompile, and in many cases
restart the Servlet engine. We realized that this would make our
applications difficult to maintain so we started looking for other
options. We eventually developed a system based on three major
requirements:

1) The HTML must be kept in a separate file from the Servlet and can be
easily changed without recompiling the Servlet.

2) The HTML must be easily edited with any of the many HTML tools
available.

3) The Servlet must be able to plug dynamic content into the HTML easily.

If Velocity had been around then, we could have saved ourselves a lot of
trouble because Velocity serves exactly this purpose.

The Advantages of Velocity

Velocity is a simple to use, open source web framework designed to be used as
the view component of an MVC architecture. You can use it as a
replacement for JSP or alongside JSP. It fits very well into more complex
frameworks such as Struts or Turbine.

The main advantage of using Velocity over JSP is that Velocity is simple to
use. The Velocity Template Language (VTL) is so constrained in its capabilities
that it helps to enforce separation of business logic from the view. You
won't see any Java classes in a Velocity template (an HTML document with some
Velocity placeholders). In fact, Velocity is so simple to use that you can
teach your HTML team how to use Velocity in just a few hours.

Installing Velocity

Assuming you already have Tomcat installed, create a new web project called
velocity and set up the appropriate WEB-INF directories. Under WEB-INF, in
addition to lib and classes directories, create a directory called
templates.

Velocity is very easy to install.You can get Velocity from the binary
download page of the Apache Jakarta Project. (http://jakarta.apache.org/site/binindex.cgi)
Download the zip or tar file. Inside you will find two jar files named,
velocity-1.3.1.jar and velocity-dep-1.3.1.jar. Extract these two files to
the WEB-INF\lib directory of your velocity web project.

That's it. Velocity is now installed. The Velocity jar files
should always be added to the lib directory for each Velocity project.

A Basic Velocity Program

Instead of going into a long, boring discussion of Velocity, let's look at a
simple example of a Velocity program.We will start by assuming that we have
created a web project called velocity in our Tomcat installation.

We will need to setup our web.xml file in our project's WEB-INF
directory. Here is what it may look like:

This is a fairly typical web.xml file. We have added one parameter
representing the properties file used by Velocity. The properties file
tells Velocity where to find the templates it will use and how to process them.
Here is a sample properties file:

The resource.loader tells Velocity that we will be getting the
templates from files. Other options such as reading them from a jar file
are also available. file.resource.loader.class tells Velocity the name of
the class to use to read the files. file.resource.loader.path tells
Velocity where to look for the templates.
file.resource.loader.cache controls whether the templates will be
cached. For performance reasons you will almost always wish to select
true. file.resource.loader.modificationCheckInterval tells Velocity
to check to see if a template has been changed in two second intervals. Setting
this option to 0 will turn checking off.

A Velocity template is simply an HTML document with some special placeholders
that Velocity will recognize and replace with data you supply in your Java
program. For our simple test, we have a template named hello.vm.(It is
standard practice to name Velocity templates with the extension of
vm.) Place the template in our WEB-INF/templates directory.

<html>
<body>
Hello $name
</body>
<html>

Sample 3. hello.vm

The $name in the document is a placeholder or reference in Velocity
terminology. When we run our Java program using this template $name will
be replaced with the data we supply in our program. Let's look at a
Velocity Servlet.

The code is actually quite simple.Our Servlet extends VelocityServlet which
is the base class for Servlets in Velocity. We override the
handleRequest() method which will be invoked when the Servlet is executed.
Beside the normal HttpServletRequest and HttpServletResponse objects, Velocity
also passes us a Context object. We will place the data we wish to use to fill
the placeholders with into the Context object.In the example, the $name will be
replaced by "Velocity Test". We must return a Template object. In
our example, we are returning the template that we created above.

Using a Java Class in a Template

That was simple so now let's look at something a little more complex.
Although we could fill our Context object with separate Strings for each
placeholder, this isn't always required. We could create a Java class with
the appropriate get methods and pass that to our template.

Let's create a simple Customer class to use in our example. The class
has set and get methods that we can use to access the private Customer
variables.

The Servlet creates a Customer object and puts it into the Context
object. It then returns the customer template. How does the customer
template access the data from the Customer object? The answer is that we
use a simple dot notation.

Each placeholder will invoke the get method from the Customer object.
For example, $customer.name will invoke the getName() method of the Customer
object.

Do you notice what is missing from the template? There is no mention
of the Java class whatsoever. Although we used $customer, that was only
because "customer" was the name used when the Customer object was placed into
the Context object. Because this template doesn't know anything about a
specific Java class, it will work with any Java class as long as it has the
matching get methods.

To test this, update your web.xml file and try running it.

Using the Velocity Template Language

VTL is fairly simple but it does provide some very useful
functionality. There is, for example, an if-then-else structure
available. The most useful VTL command is the #foreach command which
allows you to process through a Collection object. This will allow you to
display a list of data on your HTML page. For a quick example, let's add
a few invoices to the customer page.

First, we will need an invoice object. I have created a very simple
example excluding most of the information you would include in a real invoice
class. However, in addition to the normal get method for each variable, I
have added a special display method that will format the data for output.
Here is the Invoice class:

#foreach will start a loop. The loop will allow us to process
all the data in the ArrayList. $customer.invoices runs the
getInvoices() method of the Customer object which returns an ArrayList.
Each entry in the ArrayList is extracted one at a time and named
$invoice. $invoice.orderDateDisplay will run the
getOrderDateDisplay method of the Invoice object. An iteration of the loop
is terminated with the #end command. When all the entries of the
ArrayList have been processed, we will leave the loop.

Conclusions

Velocity is amazingly easy to use and to integrate into any Servlet
application. Simpler than JSPs, Velocity forces you to separate logic from
content because VTL is so limited in its capabilities. Since velocity uses
placeholders in an HTML document, it is very easy to code your HTML using a
fancy HTML editor. The placeholders simply mark the place in your document
that will get replaced with real data.

Velocity can run standalone outside of a Servlet engine if you like.
You could, for example, use Velocity to read in a file and write out a new file
with the placeholders updated. In addition, Velocity has a suite of tools
that can be used that provide additional functionality.

Overall, Velocity is an extremely nice product. Because it is so
simple and concentrates on a single aspect of your web architecture it can be
easily integrated into almost any web development project.

Automate the build process of your J2ME project with Antenna

by Matthew Phillips

Introduction

I think that most people would agree that Ant is a wonderful build tool for
Java applications. I don't always use it, but I try to. If you aren't using Ant
and would like to learn how, check out Thomas Paul's article in the March, 2002 newsletter. If you are not convinced
of it's usefulness, check out Frank Carver's series of articles on building
Small and Simple Web Applications - the Friki way (see the links at the end of
this article). I'll wait here.

If you are still here or have returned, then you may be wondering how this
all applies to that J2ME project that you are working on. If so, then I'd like
to introduce you to Antenna. All those nice conveniences that the Sun Wireless Tool
Kit (hereafter referred to as WTK) provides can be done easily from your Ant
build script.

Integrating the WTK into your build

Antenna introduces several Ant tasks that perform key functions of the WTK.
Antenna relies on the WTK to do the brunt of the work, so you will need to have
that installed. As with all optional Ant tasks, you need to let Ant know about
them in your build file. Fortunately Ant makes this easy for you:

As I wrote before, Antenna relies on the WTK to provide the underlying
functionality. It does this by using the wtk.home property to find where you
have the WTK installed. Antenna defines all tasks in a file called
"antenna.properties", which is in antenna-bin-0.9.11.jar. Ant will know what it
needs to about the Antenna tasks after you add antenna-bin-0.9.11.jar to your
Ant distribution's lib folder.

Automating your jad file

The WtkJad task writes your *.jad and manifest files so that you don't have
to. It looks like this:

The target has several useful properties. The jadfile property, which is the
only required property, tells the WTK what to name your jad file. The jarfile
controls the values of the MIDlet-Jar-URL and MIDlet-Jar-Size properties in the
jad file. The name property controls the name of your MIDlet suite. Vendor and
version controls the jad properties of the same name. The manifest property
controls the name and location of the manifest file.

The two elements of the wtkjad task are the midlet and attribute elements.
The midlet element allows you to define each MIDlet in your suite. I've shown
the name and class attributes, but there is also an icon attribute if you would
like to define an icon as well. The attribute element allows you to define any
other properties that need to be defined in your jad file. Running the above
task will result in the following jad file being built for you:

Compile and Preverify

The wtkbuild task extends Ant's javac task and adds a few properties to aid
with your J2ME code. One of the big conveniences of this task is that there is
no need to add midpapi.zip to your classpath. The task does that for you
automatically. Another important, but optional, property is the preverify
property. If you set this property to true, then the preverify process will run
after your code is compiled. There is also a preverify task and a preverify
property of the package task (explained later) so you have several points in
which you can have the preverification occur. The wtkbuild task looks like this:

Packaging your code

The wtkpackage task extends Ant's jar task and gives you a little more
control over the packaging process. As stated previously, you may preverify
during this task, but since we did that during the compile process, there is no
need to here. It needs to know the directory you want to jar, the name of the
jar file, and the name of the jad file. If you have any external libraries to
include, you may do so with the libclasspath attribute. You may also run an
obfuscator during the package process, but we'll save that for a separate task.
The wtkpackage task looks like this:

Running the emulator

Running your MIDlet suite is made incredibly easy with the wtkrun task. It
needs to know the name and location of the jadfile. It will use the default
device of the WTK, but you may also use the device attribute to control this.
The default behavior of this task is for Ant to pause while the emulator is
running. That will allow you to see anything that the emulator writes to
System.out or System.err. If you don't want this behavior, you can set the wait
attribute to false. A simple wtkrun task looks like this:

Obfuscation

When you are ready to deploy your code, you may want to obfuscate it. The
wtkobfuscate task will take care of it. It can use either the RetroGuard or ProGuard. ProGuard is the default, but you may use the
obfuscator attribute to specify which to use. In order for this task to function
properly, the jar file for the obfuscator you choose needs to either be in the
WTK's bin folder or on your class path. The task looks like this:

Use Custom Tags to Aggregate RSS Feeds into JSP-Based Web Apps

With the abundance of news and blog (Weblog) sites growing continuously, keeping track of what?s going on can be a daunting task. Fortunately, standards such as RSS (Really Simple Syndication) provide an easy way to grab content from a particular site and aggregate it into a news reader application. This means that, rather than looking for news yourself, your news reader monitors sites that you?re interested in and downloads new content as it?s published.

This is a great model and many people have taken the concept to the Web by aggregating other content on their own Websites and offering aggregation services via the Web. In this article, I?ll show you how to use JSP custom tags to implement this type of functionality as a reusable component within your own JSP-based Web applications. Although I?m assuming that you?re comfortable with Java and you have some working knowledge of building Web applications with JavaServer Pages (JSP), I won?t assume any knowledge of JSP custom tags. Before we get started, download the code we'll use in this article here.

RSS and News Aggregators - a Brief History

The RSS standard has been around for the last few years, but only recently has it really started to catch on. One of the reasons behind this is that software like MovableType and Radio Userland have made blogging available to the masses in an affordable package. Where previously, news sections of Websites contained snapshots of a particular company or individual, and were rarely updated, these RSS tools allow us to take a more dynamic approach, providing an easy mechanism by which to add new news items to sites.

Thanks to these tools, and the proliferation of news being presented via the Internet, keeping up to date is a much more difficult task than ever before. For this reason, a standard format was defined to allow news to be syndicated and aggregated through desktop applications called news readers.

The result was that RSS, the Really Simple Syndication format, was born. In essence, RSS is simply an XML document that can be used to describe the content available on a given Website. Typically, ?content? means news items, but other uses of RSS include summarising articles, short stories, and so on. A good example of an RSS feed is the UK news from the BBC. The introduction of the RSS standard format made the aggregation of content much, much easier than before.

Reading RSS Feeds from Java

Since RSS feeds are nothing more than standardised XML documents, reading and processing RSS is fairly easy in any language that provides support for XML. Now that J2SE 1.4 provides integral support for XML, it?s simply a matter of using the appropriate classes to read in the XML document. Once the document has been read, presenting it back to the user in a desktop or Web-based application is then trivial.

For the purpose of this article, I?m going to show how to aggregate content from an RSS feed into your own Web applications. The use of this technique can be applied across a wide range of applications, from corporate intranet sites aggregating content from various departments, to personal Websites aggregating content from friends and family.

Before we talk more about custom tags, let's quickly look at how we will read in the RSS feeds using Java code. Manipulating XML documents in Java code can be tedious and therefore, rather than code against the raw XML, I've chosen to build a very simple object representation of the RSS feed. The first class is called RssFeed, and this represents a given RSS feed that contains a number of items.

In reality, RSS feeds have other characteristics, but the collection of items is enough for our purposes. Next up is the Java class that represents an item in a RSS feed. Typically, several pieces of information will be presented for each item in the feed; I've chosen to wrap these within an object. The information that?s relevant for each item in this example is its title and a link (URL) back to the full story on the Web.

package rss;

public class RssItem {

private String title;
private String link;

public String getTitle() {
return title;
}

public void setTitle(String title) {
this.title = title;
}

public String getLink() {
return link;
}

public void setLink(String link) {
this.link = link;
}

}

The final piece of the puzzle is the class that will actually read the XML document and transform it into our object representation -- the RssReader class. Instead of getting bogged down with the syntax and semantics of reading XML files, the implementation of this class has been omitted. Essentially, all that the read() method does is access the RSS feed at the specified URL and convert each item contained within the feed into an RssItem object.

This is all the logic we need to read RSS feeds. Let's now take a look at how to hook this into a JSP page.

Reading RSS Feeds from JSP

For the purposes of this article, I?m going to assume that you?ve chosen to build your Web application using Java Web technologies and, specifically, that you?re going to use JavaServer Pages (JSP). One of the great things about this technology is that it makes the injection of dynamic behaviour into your pages easy -- all you need to do is add a little Java here and there. For example, we could use the classes we've just built in a JSP page:

This code grabs the RSS feed from the specified URL and, using the standard java.util.Iterator class, loops over each item displaying a hyperlink back to the full story.

While embedding Java code as a scriptlet is useful, it can soon become problematic -- particularly if you need to reuse that code on other pages within your Web applications. After all, there is no easy way to reuse such code; to simply copy and paste the code throughout your application will eventually lead to maintainability issues, as changes need to be replicated in every occurrence of that script. Ideally, we want to reuse code at the component level, taking a given component and using it wherever necessary. JSP custom tags are the answer.

Wrapping the RSS Reader in a JSP Custom Tag

JSP custom tags are a means to wrap common and recurring logic into a form that?s reusable across the pages of your Web applications. In the current version of the JSP specification (1.2), the behaviour of a custom tag is implemented as a Java class that activates a specific interface, in much the same way you?d implement a servlet, for example. This Java class is generally called the tag handler.

The key difference between custom tags and regular Java classes lies in the way they?re used within JSP pages. Where Java code is simply embedded in the page, custom tags are used with an XML syntax. For example, here?s how the custom tag will be used when we?ve finished building it. Notice that here we have a starting tag, some body content, then the ending tag.

This will achieve exactly the same result as the scriptlet code we saw before -- the tag provides the iteration over each item in the feed. Essentially, the tag is now the looping construct and the body content of the tag is evaluated for each iteration, in the same way that the content of a while loop is evaluated for each iteration. Custom tags provide us with a way to build cleaner, more concise JSP pages that benefit from the advantages of reusing components -- increased maintainability, quality, reliability, and so on.

Building a Tag Handler Class

Now that you have an understanding of why custom tags are useful, let?s take a look at how to build one. As I said before, the behaviour is wrapped up as a regular Java class that implements a specific interface. In the case of custom tags, the interface that must be implemented (at a minimum) is Tag, from the javax.servlet.jsp.tagext package. This interface is fairly straightforward and provides a number of callback methods that will be executed when the tag is used on a JSP page. At runtime, the JSP page creates for a given custom tag a new instance of the tag handler class, and calls the callback methods. This might sound complicated at first, but it becomes fairly straightforward once you?ve built a couple of tags for yourself.

For convenience, the JSP specification also provides a basic implementation of this interface, called TagSupport, which is an ideal starting point from which to build your own tags. The following code snippet shows the start of the source code for the tag handler, including all the necessary imports. It also shows a couple of attributes -- we?ll see where these are used later on.

In the tag usage example I presented earlier, we saw that there was a starting tag, some body content, and the ending tag. These different aspects of the tag are important because they define when the JSP page will fire the callback methods that we write in our tag handler class. The functionality we need to implement is identical to that shown before; we need to read in an RSS feed from a URL, then iterate over each item in the feed so that a hyperlink can be generated. With the TagSupport class, the three callback methods that are available to us are doStartTag(), doAfterBody() and doEndTag(). Let?s look at each of these in turn.

The doStartTag() method is called when the starting tag is encountered on the JSP page, and is called only once for any given custom tag.

Since this code is only called once per tag, and is the first of the callback methods to be fired, it?s here that we can read the RSS feed and set up an iterator to loop over the collection of items. If you think of this method as the start of the while loop, the first step is to check that there are items in the collection. If there are, we want to make the first item available for use in the JSP page, so that we can utilise it within the body content of the tag. To do this, we set an attribute on the JSP PageContext to refer to the first item in the collection. Next, we must tell the JSP page that the body content of the tag should be evaluated, which is achieved by returning a constant value of EVAL_BODY_INCLUDE. If, however, there aren?t any items in the collection, we tell the JSP page that it should skip evaluating the body content by returning the constant value of SKIP_BODY.

The next callback method of interest is the doAfterBody() method, which is called after the body content of the tag has been evaluated.

Once the body content is evaluated, the next step is to see whether there are any more items in the collection. If there are, we want to make the new item available to the JSP page and indicate that the body content should be re-evaluated, by returning a constant value of EVAL_BODY_AGAIN. After this, the doAfterBody() method is called again, to see if yet another evaluation is required. This sequence repeats until there are no more items in the collection, in which case a constant value of SKIP_BODY is returned.

When all the evaluations have been performed, the final callback method is executed.

public int doEndTag() throws JspException {
return EVAL_PAGE;
}

This implementation simply tells the JSP page that the remainder of the page should be processed as normal. In fact, the implementation of the TagSupport class contains the same implementation of this method, meaning we don?t actually need to implement it. However, I?ve shown it here for completeness.

The final method we need to implement is a setter for the URL. If you look back to the example of how the tag will be used on the page, you?ll notice that the URL of the RSS feed is specified as an attribute of the custom tag. To make this information available to the tag handler class, we need to write a setter method in the same way we?d write setter methods for JavaBean properties. Any setter methods that correspond to attributes of a custom tag are called before the callback methods, so that their values are available for use within those callback methods.

public void setUrl(String url) {
this.url = url;
}

}

Compiling the Tag Handler Class

To compile the tag handler class, make sure you have the Servlet and JSP classes in your classpath. If you?re using Tomcat 4, these can be found in the $TOMCAT_HOME/common/lib/servlet.jar file. The resulting class file should be placed under the WEB-INF/classes directory of your Web application, in the appropriate directory/package structure.

Describing the Tag

The next step is to describe the custom tag using an XML file called a tag library descriptor, or TLD file. This step is necessary because it allows us to define how the custom tag will be used on the page, the name of its attributes, and so on. Starting at the top, we have all the usual XML header information:

Next, we have the start of the tag library definition. Although custom tags are reusable, they must be defined within the context of a tag library -- a collection of one or more tags that are typically related in some way. This block of XML allows us to define the version of our tag library, the required version of JSP, a short name, and a description of the tag library.

Following this is the definition of the tags in the tag library. Here we have only one, which defines the name of the tag as it will be used on a JSP page, the name of the tag handler class, the body content type and, again, a short description.

<tag>
<name>rssFeed</name>
<tag-class>tagext.RssFeedTag</tag-class>
<body-content>JSP</body-content>
<description>
A tag to present a headlines/titles of items from an RSS feed.
</description>

Of these, the body content probably needs some more explanation. The JSP specification provides several body content types that can be defined for custom tags, with the two most useful being empty and JSP. A body content type of empty indicates that the custom tag will be used without body content on the page, which is handy when you simply want the tag to perform some sort of action. On the other hand, a body content type of JSP indicates that there will be regular JSP constructs used between the start and end tags. This is what we?re using in this example, because we?d like the body content to be evaluated for each item in the RSS feed.

The next part of the XML file describes the scripting variable that will be introduced into the page within the body content of the tag. In the tag handler code, we get the next RSS item from the collection, then place the reference to that object in the page context under the name rssItem. One of the things custom tags can do is make these attributes available as scripting variables on the JSP page, so they can be accessed with the request-time expression syntax of <%= ? %>. Here, we specify the name and type of the variable, along with a scope of NESTED to indicate that the variable should only be accessible between the starting and ending tags.

The final aspect of the tag to describe is its attributes. In this example there is only a single attribute, called url, which is used to indicate the source of the RSS feed. To ensure that the tag works as expected, we've stated that this attribute must be supplied when the tag is used. The rtexprvalue element of the attribute tag says that the value of the attribute must be statically defined in the JSP page. In other words, the value of the attribute isn't the result of a request-time expression.

Using the Tag

For the purposes of this example, let's assume that the TLD file has been saved as rss.tld under the WEB-INF directory of your Web application. To use a custom tag, you first need to tell the JSP page where to find the description of that tag. This is achieved through the taglib directive, with the uri attribute pointing to the TLD file that represents the tag library, and the prefix attribute stating how the tags in that tag library will be identified and used on the page. Then, using the same syntax as before, we can use the tag to read the RSS feed provided by any Website, and generate a set of hyperlinks to the current news stories on that site.

Future Enhancements

The tag presented here is fairly simple in its implementation, and there are many enhancements that could be made. For example, every time the JSP page is requested, the tag opens up an HTTP connection to retrieve the contents of the RSS feed. While this is okay for a low traffic site, a better solution would be to cache the feed on a regular basis. This would avoid the performance penalty associated with opening a network connection for every page request.

Also, the tag doesn't take into account what happens if a network error occurs. For example, the Website might be down or may not be functioning correctly. In any case, you would probably want to add some error handling, perhaps to display a message to alert users that the feed isn't currently available.

Summary

In this article we've looked at what RSS is, how to read RSS feeds, and how to integrate this functionality into a JSP-based Web application. Although we could have built this functionality directly into the JSP page using Java code scriptlets, developing a JSP custom tag has allowed us to build a more maintainable component with the added advantage that it?s reusable, too.

Building the tag handler class and writing the TLD file does involve slightly more work than would embedding Java code into the page. However, I believe that the benefits in maintainability and reusability easily justify the additional effort involved.

Asynchronous queries in J2EE

There's a servlet design problem so common that I hear it
asked at least once a week, if not more often, in customer meetings, on Java
newsgroups, or on forums like the JavaRanch or jGuru. The question, as it's
often phrased is:

"How do I get my servlet to stop timing out on a really long
database query?"

Regardless of how much database tuning effort you put into a
project, there usually are one or two queries that end up taking much, much
longer than you hope. In many cases, these especially complex queries may take
up to a minute or longer to execute. The issue is that if you call a long
query directly from a Servlet or JSP, the browser may time out before the call
completes. When this happens, the user of your application will not see the
result. There are commercial solutions to this problem (such as Async Beans in
WebSphere Application Server, Enterprise Edition) but if you do not have access
to those proprietary solutions, or need a solution that will be portable across
multiple application servers, you are left to your own devices.

So, given this problem, developers often start casting about
for a solution and usually end up finding out about using the <META> tag
for server polling. This tag tells the client it must refresh the page after a
number of seconds[1].
The number of seconds and the URL of the new page are specified in the CONTENT
attribute of the tag:

<META http-equiv="Refresh" content="10; url=New-page.html">

So, once a developer determines that he can have the browser
query the servlet on a regular basis to re-fetch a page, they quickly determine
that the servlet can check for the value of a variable (perhaps in the
HttpSession) to determine if the page returned will contain either the that the
user requested, or a <META> tag that will display a "Please wait?"
message and try fetching the page again later.

What goes along with this solution is usually a decision to use
a separate thread to handle the long query so that the main thread can return
the HTTP response as quickly as possible. While this has been a perfectly
acceptable solution in the past, it introduces complications, both from a
programming perspective, and from a specification perspective today. For
instance, in WebSphere Application Server 5.0, if you attempt to access a
database on a thread you have spawned yourself, you will see the following
errors appear in the log:

According to a Technical note from WebSphere support "If a
Servlet is spinning its own threads and accessing a database, the J2EE
specification is not clear on this, so WebSphere Application Server 5.0 will
allow it at this time. IBM is working with Sun to clarify this in the
specification, so eventually (i.e. J2EE 1.4) spun threads from a Servlet
accessing a database outside of a transaction will not be supported either".
Later the same tech note states "Customers should consider changing their
application to comply with the J2EE specification."

However, this is not the only problem that comes up with
starting your own threads from a servlet. Not only will the transaction
context not be defined, but the security context will not be defined either. Also,
if the JVM running this thread dies, then the query is lost; even if your
HttpSession is persistent, no other application server JVM in the cluster will
know about the query that was running on the thread. In general, given the
direction in which the J2EE specifications are heading, toward becoming more
restrictive regarding thread creation, it's probably not a good idea to create
your own threads from a Servlet in any case.

A J2EE-compliant solution for Queries

So, how can we manage to execute a long-running query
asynchronously from a servlet that is intended to return the result of that
query, and yet remain firmly within the bounds of good J2EE design? What you
should examine instead of creating your own threads is using your application
server's built-in internal JMS messaging (such as WebSphere's based on
WebSphere MQ) to do the database access within the context of a Message-Driven
Bean, and then have the MDB post a response to another queue which the Servlet
would then query for the particular response. The way it works is that your
servlet posts a "request" on a queue and returns to the user a page
that then checks back later, either using a <META> tag or a Javascript
script. When "later" happens, the servlet looks for the response in
the queue and shows the result.The following diagram illustrates this process.

The pages returned by both the Requesting servlet and the
Displaying Servlet (prior to the final display of the results) use the META tag to automatically request a refresh of the Displaying Servlet every few seconds.
The browser polls the Displaying Servlet; each time it is invoked, the servlet
checks for a result inside the JMS Queue.

The Query Processor then takes in the requests, executes the
appropriate long-running database code, and then returns the response to the
servlet through another queue. Query Processor should be implemented as a
Message-Driven Bean, which has several advantages ? first, since MDBs are EJBs,
they tie together in a single transaction the receipt of the message on the
queue and the processing of any database updates that execute within the onMessage() method. This means that
if a database failure occurs that prevents processing of a request, that the
request message will be placed back on the queue so that it may be tried again
later; this helps with resolving transient errors like a database connection
not being available. Second, MDB's within a cluster listening on the same
queue act as "Competing Consumers" [Hohpe]. This means that all of the
members of the cluster balance work of processing messages; if messages are not
being processed fast enough, you can add more JVM's to the cluster in order to
increase the processing power available.

Implementing multiple Queries with Commands

An important point to examine here is how you can implement your
MDB in such a way that you don't need to have a different queue or MDB for each
potential query that you run. For example, the simplest implementation of this
pattern would simply involve pulling parameters off of a message, and then
directly invoking a DAO with the parameters thus obtained. So, your
implementation might look something like this:

publicvoidonMessage(javax.jms.Messagemsg){

TextMessagetMsg=(TextMessage)msg;

StringcustomerNumber;

try{

customerNumber=tMsg.getText();

}catch(JMSExceptione){

System.out.println("Exception caught: "+e);

fMessageDrivenCtx.setRollbackOnly();

return;

}

OrderDAOorderDAO=newOrderDAO();

Collectionorders=orderDAO.findOrdersBy(customerNumber);// now send the response back by looking up the

// response
queue and returning the collection...

}

However, the problem with this approach is that if we have
another long running query (say, finding all Orders by the items they contain)
then we'd need to write and deploy another MDB to invoke a different method on
the DAO; something that would quickly become tedious as the number of queries
increased. So, instead, a better approach is to use the Command pattern
[Gamma]. In fact, this is also the "Command Message" pattern [Hohpe]. Our
solution is as follows;

(1)Create a Command
object in your Requesting Servlet that represents the particular query that you
want to execute

(2)Send the Command
object as an object message to the command processor MDB

(3)Invoke the execute()
method on the command object, letting it execute its query inside the context
of the MDB

(4)Obtain the result from
the Command object and package it as a result on a response queue.

So, a more sophisticated example of our MDB, one that deals
with as many query types as you may care to support, would look like the
following:

publicvoidonMessage(javax.jms.Messagemsg){

Responseresponse;

try{

ObjectMessagemessage=(ObjectMessage)msg;

ProcessingRequestrequest=(ProcessingRequest)

message.getObject();

response=request.execute();

}catch(Exceptione){

response=newErrorResponse(e);

}

processResponse(response,msg);

}

To make this work, we need to implement our Commands so that
they execute their long-running queries in response to the execute() message. This is a standard
implementation of the Command pattern. In our case, let's consider just one
of many possible Commands, an OrdersForCustomerRequest,
as shown below:

Creating Commands in a Servlet

In order to understand how this Command will be used, let's
begin at the Servlet that creates it, the OrderQueryRequestServlet:

This servlet would run in response to a PST request form an
HTML page that includes one form parameter, the customerNumber that we are
querying for. The flow of the servlet is simple; it obtains the parameter,
creates a Command object (the OrdersForCustomerRequest)
that will use that parameter, and then calls the packageRequest()
method to manage the request. The packageRequest()
method is the one that actually does the heavy lifting of placing that
command on the request queue. Its implementation (from the superclass DeferredDatabaseAccessServlet) is
shown below:

publicvoidpackageRequest(ProcessingRequestreqObj,

HttpServletRequestservletReq){

try{

//Create a correlation id

StringcorrelationId=Long.toString(System.currentTimeMillis());

//Look
up the QueueConnectionFactory in JNDI

javax.jms.QueueSessionsession=

conn.createQueueSession(

false,

javax.jms.Session.AUTO_ACKNOWLEDGE);

javax.jms.QueueSendersender=

session.createSender((Queue)requestQueue);

javax.jms.ObjectMessagereplyMsg=

session.createObjectMessage(reqObj);

//
Now need to set the correlation ID

replyMsg.setJMSCorrelationID(correlationId);

replyMsg.setJMSReplyTo(responseQueue);

//
Finally send the message

sender.send(replyMsg);

session.close();

// Now set the correlation id in the session
so

// the response servlet can find the message

HttpSessionservletSession=servletReq.getSession();

servletSession.setAttribute("MessageCorrelationId",correlationId);

}catch(Exceptione){

System.out.println("Could not
process Response "+e);

}

}

Now we begin to get to some of the more interesting design
points about implementing this pattern. Note that in this method we have used
a JMS correlation id to uniquely identify this message. Correlation identifiers
(as described in [Hohpe] and [Monson-Haefel]) are used to link related messages
together. The most common use is exactly what we are doing here: to link a
response with an earlier request, so that you can locate a particular response
out of many unrelated responses on a shared queue. We'll return to this later
after we trace the path of this particular command object through the rest of
the system. Note that in this case we are simply using the current system time
as our correlation identifier. If this application runs in a cluster (or just
runs very quickly) it is possible, but improbable, that the same correlation
identifier might be used for two messages. In a real system using this
pattern, you should probably also append a random number to your correlation id
to prevent this.

At this point, the Command object for our query has been
placed on the outgoing (request) queue and awaits processing. After it has
been placed on the queue, the requesting servlet places the message's
correlation id in the HttpSession
(we'll see why later) and then returns a page to the user that asks him to
wait. This page is shown below:

As you can see, this page uses the meta tag described above
to check in on the Servlet named ResponseServlet
every 3 seconds. We'll visit this servlet later, but for now let's follow the
trail of our Command message.

Processing Commands in an MDB

The next stop for our Command object is the MDB that
processes all of the commands placed on the request queue. In our example,
this is the DatabaseAccessProcessorBean.
As with all MDB's, the action begins in the onMessage()
method, where the message is received:

publicvoidonMessage(javax.jms.Messagemsg){

Responseresponse;

try{

ObjectMessagemessage=(ObjectMessage)msg;

ProcessingRequestrequest=(ProcessingRequest)message.getObject();

response=request.execute();

}catch(Exceptione){

response=newErrorResponse(e);

}

processResponse(response,msg);

}

Here the first thing that happens is that the Command object
is extracted from the ObjectMessage
created in the servlet. Once the Command object has been obtained, then the
Command's execute() method can
be invoked. Remember that execute()
is polymorphic; in our case since this Command is an instance of the OrdersForCustomerRequest class, then
this is the execute() method
that runs:

publicResponseexecute(){

// Note ? this sleep() is ONLY to simulate
a long running query!

// Do NOT do this in your own code!

try{

Thread.sleep(10000);

}catch(InterruptedExceptione){

}

// The method as it should be implemented really
starts here.

Responseresponse;

try{

OrdersForCustomerResponseaResponse=newOrdersForCustomerResponse();

aResponse.setCustomerNumber(customerNumber);

Collectionresults=getOrdersFor(customerNumber);

aResponse.setOrders(results);

response=aResponse;

}catch(OrderExceptione){

ErrorResponseaResponse=newErrorResponse(e);

response=aResponse;

}

returnresponse;

}

Before moving much farther into this method, I have a
confession to make. It's hard to simulate a long-running query in an example
like this. Usually queries take a long time to execute because they are examining
a large number of tables, or a large number of rows. In any case, this article
is about how to deal with long running queries when they occur, not how to
create a long-running query. So, in this example, I slowed down the execution
of the Command by adding a Thread.sleep()
for 10 seconds. This is not something you should normally do; in fact, doing
this (or any sort of thread manipulation) is frowned upon by the EJB
specification, even though it will execute in WAS 5.0.

Now that we're past that nastiness, though, we get to how
the Command really runs. In this case, we construct a Response object (of the
type OrdersForCustomerResponse),
and then run our query. I'll not show the JDBC code to execute the query here,
but if you are interested, you can download the example and look at the code.
The point is that this is where you execute the long running query. The
advantage of doing it here is that the EJB container is managing the thread in
which this query runs, and what's more, you have control over whether or not
you choose to execute the query within an EJB transaction (in which case the
transaction could time out before the query completes) or outside of an EJB
transaction within a Resource Manager Local Transaction (see the InfoCenter or
Chapter 29 of [Brown] for information on RMLT's).

After the query completes, the collection of results is
placed in the Response object, and is returned back to the onMessage() method, which then calls
the processResponse() method,
shown below:

privatevoidprocessResponse(Responseresponse,MessageinMsg){

try{

InitialContextctx=newInitialContext();

Objecto=ctx.lookup(

"java:comp/env/jms/QueueConnectionFactory");

qcf=

(QueueConnectionFactory)javax.rmi.PortableRemoteObject.narrow(

o,

QueueConnectionFactory.class);

//Create a QueueConnection on the Connection Factory

//and this queue

qConn=qcf.createQueueConnection();

DestinationreplyQueue=inMsg.getJMSReplyTo();

//Look
up the QueueConnectionFactory in JNDI

javax.jms.QueueSessionsession=

qConn.createQueueSession(

false,

javax.jms.Session.AUTO_ACKNOWLEDGE);

javax.jms.QueueSendersender=

session.createSender((Queue)replyQueue);

javax.jms.ObjectMessagereplyMsg=

session.createObjectMessage(response);

//
Now need to set the correlation ID

replyMsg.setJMSCorrelationID(

inMsg.getJMSCorrelationID());

//
Finally send the message

sender.send(replyMsg);

session.close();

qConn.close();

}catch(Exceptione){

System.out.println("Could not process Response "+e);

}

}

There isn't much new here, since this is nearly the same as
the corresponding packageRequest()
method in the DeferredDatabaseAccessServlet
we saw earlier. Basically this method exists to place the response object in
an ObjectMessage, and then place
that ObjectMessage back on the
response queue. The only two points of interest in this method are:

(1)Instead of hard-coding
the queue name we've used the JMS reply-to queue feature, which, you'll recall,
we set up in the packageRequest() method
earlier. (By the way, this is called the "Return Address" pattern in [Hohpe].

(2)Also, we set the
correlation id of the response message to be the same as the correlation id of
the request message. This is what will enable our Response servlet to locate
the right response method from the queue out of all the responses placed on the
queue.

So, now that the response has been placed on the response
queue, we're ready to examine our Response servlet, and determine how it
handles looking for the response.

Checking for Responses

You'll remember that the <META> tag will poll the ResponseServlet every three seconds to
see if a response has been returned. That polling invokes the following doGet() method:

publicvoiddoGet(HttpServletRequestreq,HttpServletResponseresp)

throwsServletException,IOException{

try{

ResponseprocessingResponse=checkForMessages(req,resp);

req.setAttribute("receivedResponse",processingResponse);

RequestDispatcherdisp=

getServletContext().getRequestDispatcher(

getResponseURL(processingResponse));

disp.forward(req,resp);

}catch(ProcessingExceptione){

log("ProcessingException +
"+e);

RequestDispatcherdisp=

getServletContext().getRequestDispatcher(

"/Error.html");

disp.forward(req,resp);

}catch(NoRequestExceptione){

log("NoReplyException + "+e);

RequestDispatcherdisp=

getServletContext().getRequestDispatcher("/Error.html");

disp.forward(req,resp);

}catch(ReplyTimeoutExceptione){

RequestDispatcherdisp=

getServletContext().getRequestDispatcher("/PleaseWait.html");

disp.forward(req,resp);

}

}

This method begins by checking for messages from the queue.
We'll look at that method in a moment, but for the time being, let's just
examine what happens after the method checkForMessages()
runs. That method will either return a response, or throw one of several
possible exceptions. Obviously, if a problem occurs (indicated by a ProcessingException) then that must be
handled appropriately by displaying an error page. The more interesting case
is where the checkForMessages()
method throws a ReplyTimeoutException,
which is what happens when the reply message has not been received on the
response queue. In that case, it redisplays the Please Wait page shown above,
and the browser will then re-query the response servlet 3 seconds later. If a
Response is returned, it will locate the appropriate URL of the JSP or Servlet
to display those results and forward to that URL.

So, now that you understand how responses are handled, let's
look at the implementation of the checkForMessages()
method and see how it checks the queue for responses.

publicResponsecheckForMessages(HttpServletRequestreq,

HttpServletResponseresp)

throwsProcessingException,NoReplyException,ReplyTimeoutException{

ResponseresponseObj=null;

try{

HttpSessionhttpSess=req.getSession(true);

Stringid=(String)httpSess.getAttribute("MessageCorrelationId");

if(id==null)

thrownewNoRequestException(

"Not Waiting on a Message");

QueueSessionsession=

conn.createQueueSession(

false,

javax.jms.Session.AUTO_ACKNOWLEDGE);

Stringselector="JMSCorrelationID = '"+id+"'";

QueueReceiverrcvr=

session.createReceiver(responseQueue,selector);

MessagereceivedMsg=rcvr.receiveNoWait();

if(receivedMsg==null){

thrownewReplyTimeoutException(

"No Reply Message Received");

}else{

ObjectMessageinMsg=(ObjectMessage)receivedMsg;

responseObj=(Response)inMsg.getObject();

}

session.close();

returnresponseObj;

}catch(JMSExceptione){

thrownewProcessingException(e);

}

}

The first possible thing that can go wrong in this method is
that the ResponseServlet has
somehow been invoked when this user does not have any outstanding requests.
The first few lines look in the User's HttpSession
for an identifier named "CorrelationID", which should have been placed there by
the servlet that originally made the request. If it is absent, then this is an
error, and the NoRequestException
is thrown.

After guaranteeing that there is an outstanding request for
this user, then what happens is that the method creates a QueueReceiver on the response queue
and then sets a message selector on the Queue. The message selector in JMS
(see [Monson-Haefel]) is an implementation of the "Selective Consumer" pattern
from [Hohpe]. What it does is to only return back those messages on the queue
that have headers that match the pattern entered as the message selector. In
this case, we are scanning for messages that have a correlation identifier
matching the one held in the HttpSession
(e.g. only responses for the last request made by this user). If there is a
message on the queue matching that query, it will be returned from the receiveNoWait() method. If not, that
method will immediately return null. A null return value means that the
request is still being processed, so the method then throws the ReplyTimeoutException.

This is the last method of our implementation to review, and
it brings up the last JMS design point to consider. In our design we've chosen
to use a fixed request queue and a fixed response queue combined with a message
selector on the response's correlation id to find only those messages that
belong to a particular user. Another option would have been to use JMS
temporary queues. In that case, we could have created a temporary queue for
the response back in the packageRequest()
method and then placed that temporary queue in the HttpSession. The problem with that approach is that it
won't survive failure of the JVM in which the message was received. Since JMS
temporary queues are not serializable, if the HttpSession
is lost then the queue (and all of its messages) are lost as well. The message
selector approach will work regardless of whether or not the JVM that receives
the response is the same as the one that originally placed the request, since
the message selector (being only a string) is easily serializable.[3]
One thing that your designs should deal with, however, is what to do in case
of VERY long queries, where the time it takes to execute the query is greater
than the session timeout value, or when the user loses interest and closes the
browser prior to the display servlet being able to fetch back the response
message from the queue. [Hohpe] discusses strategies for clearing queues in
situations like this.

Finally, there is an additional downside to the message
selector approach. Many JMS providers (such WAS embedded messaging) do not
have indexing capabilities. For a large number of queries, the use of a
message selector could lead to poor performance due to the inefficiency of
selecting out the required message. Thus, there's a potential tradeoff between
performance and scalability that you need to carefully evaluate.

An Alternate User Interface Choice

What we have shown in this article is a mechanism for
allowing each user to have a single outstanding request that is placed on a shared
queue. However, what happens if you have not one, but several very long
queries? And what if your users would like these queries to be able to run
simultaneously instead of one at a time? Luckily, our basic Command-driven
design can be adapted to solve that problem as well. In our design we've
chosen to place a single correlation id in the HttpSession. However, it's not
much of a stretch to imagine that you could instead place a collection of
correlation id's in the HttpSession, with each one representing an independent
query. In that case, what you would need to implement (instead of a single
"please wait?" page) would be a page that shows the status of several queries
at once. As the results are returned on the queue, the polled servlet that generates
that page would add the completed query results into the HttpSession as well.
The servlet would then indicate by a link which queries are completed, and thus
available for review, and which ones are still in progress.

I will warn you, however, that if your problems are getting
this sophisticated, that you should probably look into a commercial workflow
tool like the WebSphere Process Choreographer (part of WebSphere Enterprise
Edition) to solve this problem. At this point this is no longer a simple issue
of avoiding threads; it's become a more complex problem.

Summary

In this article we've covered a lot of ground, but in
essence only conveyed one simple idea. Rather than spinning off your own
threads to handle long-running queries, it's best to let the container manage
the threads by taking advantage of JMS messaging and Message-Driven Beans.
Hopefully you now have the tools necessary to build your own systems using this
approach, and can take advantage of the benefits of letting containers manage
your threads.

Acknowledgements

Thanks to Bobby Woolf for his insightful comments in
improving this article, and for catching a couple of technical errors as well.
Also, thanks to Roland Barcia and Wayne Beaton for their suggestions for
improvement to the article.

It's where you come to learn Java, and just like the cattle drivers of the
old west, you're expected to pull your weight along the way.

The Cattle Drive forum is where the drivers
get together to complain, uh rather, discuss their assignments and encourage
each other. Thanks to the enthusiastic initiative of Johannes de Jong, you can keep
track of your progress on the drive with the Assignment Log. If you're tough enough
to get through the nitpicking, you'll start collecting moose heads at
the Cattle Drive Hall of Fame.

Howdy, y'all. Glad to see that everyone made it through the holidays!
Now it's time to take a look around and see where we are at the end
of the beginning of the new year! (Kind of a state-of-the-drive address.)

The winter snows have diminished somewhat, but in places the drifts
are still so high you could hide a full-grown giraffe, and elsewhere
the rains have left mud so deep them poor doggies are practically
swimming.

Despite the climatic unpleasantness, activity has picked up a
little bit on the drive. We now have 14 drivers currently active,
and everyone seems to be making headway.

Another moose on the wall for...
The most notable milestone this past month was reached by
Greg Neef, who bagged his third moose for passing Servlets in stealth
mode. Nearly went unnoticed! If it weren't for the sharp eyes of
Michael Matola, we still might not know. Pauline McNamara came back
from regions unknown, and is closing in on the end of JDBC. Good to
see you, Pauline.

Jeremy Thornton and Kate Head are struggling up that sharp incline
at the end of the Oop section of the trail. If they keep at it
and the weather cooperates, they should be breaking through the pass
and out into the broad plains of Servlet country pretty darn soon.
Most of our drivers are scattered along this part of the trail. The
terrain here is pretty spread out, and this part of the drive seems
to swallow folks up for a while. Just keep an eye open, you're
sure to see a few drivers along this stretch. Give 'em a nod and wish
them luck!

And the intrepid souls who have just started out on the first leg of
the drive are all making steady progress. A few folks have dropped out
of sight, but once the rain stops and the periods of sunlight grow
longer, I'm sure they'll be popping back into view. I know I'm longing
for longer days and more time to roam on the ranch.

Michael has resumed the duty of updating the log, (thanks Michael!)
after his holiday break, and I try to drop in as often as possible.
Still waiting for the XML trail to open up so's I can get back to
moving cattle.

Nitpicking is hard work
too...We know they're workin' reeeeeally
hard, after all, we've seen what those assignments look
like once the nitpickers have combed through 'em. Hats
off to Marilyn deQueiroz, Pauline McNamara, and Jason Adam for their dedication
and patience with the pesky vermin that always manage to
make their way into those assignments.

Joinin' the Drive
You think ya got what it takes? You ready for some work and some good learnin'? Then pull up a stool and read up on Joinin' the Drive. Good luck!

Lookin' forward to seeing you out on the range! Keep your eyes on the
trail, but don't forget to look up every now and again to enjoy the
scenery!

A lot of IT people are busy making different applications running on different platforms "talk" to each other. XML was invented as the "Esperanto" of the IT world to get these systems to understand each other. In practice however it just isn't that simple, as most of these applications don't talk XML yet, until this book.

This book is a real "do it" book. It does not teach you XML or XSLT but shows you how to use it. What I especially liked is that he discusses his design considerations; he wants you to understand the why's. Once he thinks you know the basics he goes back to his basic design and improves it, to make it make it fully reusable and modular making it even better.

Mr. Rawlins gives you toolbox of utilities, with the source code, that can become the building blocks for your own application integration system. I have not come across a book with as much usable code in my IT career. We have already redesigned quite a few of our systems because of it. If you are into "connectivity" you can't be without this book.

P.S. The word Legacy in the title does not imply big mainframes.
(Johannes de Jong - Bartender, January 2004)