Clustering and Load Balancing in Tomcat 5, Part 2

This is the second part of a series on clustering and load balancing in Tomcat 5 server. In part 1, I provided an overview of large-scale J2EE system design as well as various factors to be considered when designing the system for scalability and high availability. I also discussed Tomcat's support for clustering, load-balancing, fault-tolerance, and session-replication capabilities. In this part, we'll cover the architecture of a proposed cluster setup and go over the installation and configuration details in deploying the cluster (by running multiple Tomcat server instances).

Proposed Cluster Setup

Listed below are the main objectives that I wanted to achieve in the proposed Tomcat cluster:

The cluster should be highly scalable.

It should be fault-tolerant.

It should be dynamically configurable, meaning it should be easy to manage the cluster declaratively (changing a configuration file) rather than programmatically (changing Java code).

It should provide automatic cluster member discovery.

Fail-over and load-balancing features for session data with in-memory session state replication.

Pluggable/configurable load-balancing policies.

Group membership notification when a member of the cluster joins or leaves a group.

No loss of message transmission through multicast.

Clustering should be seamless to the web application and the server. It should provide both client and server transparency. Client Transparency means that the client is not aware of clustered services or how the cluster is set up. The cluster is identified and accessed as a single thing rather than individual services. And server transparency means that the application code in a server is not aware that it's in a cluster. The application code cannot communicate with the other members of cluster.

Four instances of Tomcat server were installed to set up the clustering environment. Tomcat was used for both load balancing and clustering requirements. The cluster setup was done using vertical scaling method (multiple Tomcat server instances running on a single machine). One server group and two clones were configured in the cluster (a server group is a logical presentation of the application server). The clones had same exact configuration (in terms of web application directory structure and contents) as the server group to optimize session replication. To follow are the main components in the proposed cluster setup:

Load Balancer: A Tomcat instance is configured to distribute the traffic among cluster nodes. This instance is given a code name TC-LB.

Clustering: Three Tomcat server instances were run as part of the cluster. The code names for these instances are TC01, TC02, and TC03.

Session Persistence: In-memory session replication was chosen as the session persistence mechanism. The session data was copied to all three cluster members whenever the session object is modified.

Fail-Over: The balancer application that comes with Tomcat installation is not designed to handle fail over. I wrote a utility class called ServerUtil to check the server status before forwarding any requests to it. It has two methods to verify the status of a cluster node. In the first method, it uses McastService to check if a specified server instance is currently running or not. The second method verifies a cluster node's availability by creating a URL object based on the web page URL passed in as a parameter. To use this class make sure catalina-cluster.jar (located in %TOMCAT_HOME%/server/lib directory) and commons-logging-api.jar (%TOMCAT_HOME%/bin directory) files are specified in the classpath.

The application architecture diagram in Figure 1 shows the main components of the cluster.

Figure 1. Tomcat cluster architecture diagram

Installation & Configuration of Tomcat Instances

Table 1. Hardware/software specifications of the machine used to set up Tomcat clustering.

Processor

HP Pavilion Pentium III with 800 MHz

Memory

512 MB RAM

Hard Disk

40 GB

Operating System

Windows 2000 server with Service Pack 4

JDK Version

1.4.0_02 (Note: JDK version 1.4 or a later version is required to enable Tomcat Clustering)

Tomcat Version

5.0.19

Tools Used

Ant 1.6.1, Log4J, JMeter, JBuilder

Main elements of cluster framework

Java Classes

BaseLoadBalancingRule

This is an abstract class created to encapsulate the common rules logic in the custom Rule classes. The custom load balancing rules used in the sample web application extend this base class.

RandomRedirectRule

This class defines the logic to forward the web requests to an available server in a random manner. It uses current system time as seed to generate random numbers.

RoundRobinRule

This class defined the load-balancing logic based on a round robin rule. When a request comes in it forwards the request to the next member in the list. It uses a static variable to track the next available cluster member and increments this value by one for every new request.

ServerUtil

A utility class was created to check if a specific cluster node is available or not, to receive requests. This class uses McastService (org.apache.catalina.cluster.mcast package) to detect if a cluster member has left the group.

The relationship of these Java classes is shown in the class diagram in Figure 2.

Figure 2. Cluster application class diagram

Configuration Files

server.xml

This file is used to configure clustering in a Tomcat server instance. The version that comes with Tomcat installation has the configuration details included in the file but they are commented out.

web.xml

This configuration file is used to specify that a web application session data needs to be replicated.

rules.xml

This file is used to define the custom load-balancing rules. This is the file we use to specify which load-balancing rule we want to use to distribute the load among the cluster members.

Scripts

test.jsp

A simple test JSP script was used to check the server status. It displays the system time and name of the Tomcat instance from which it's launched.

testLB.jsp

This is the startup page in our sample web application. It forwards the web requests to a load balancer filter using HTML redirect.

sessiondata.jsp

This script is used to verify that no session data is lost when a cluster node goes down. It displays session details and also has HTML fields to manipulate HTTP session object.

build.xml

An Ant build script was created to automate the tasks of starting and stopping Tomcat instances (the latest version of Ant 1.6.1 was used to run this script). Once a Tomcat instance has started successfully, you can call the test.jsp to verify which Tomcat instance is running by specifying IP address and port number. The JSP page displays the current system time and the name of the Tomcat instance. (You need to change the home directories for Tomcat servers specified in build.properties file to run the script in your own environment).