EAI: A business issue for IT

Enterprise Application Integration (EAI) is a booming business. Wintergreen
Research estimates that $500 million was spent on EAI tools in 1999, that $900
million will be spent in 2000, and that EAI spending could reach $7.3 billion
by 2003. Wintergreen also estimates that half of the world's 1999 IT labor budget
was spent on developing custom EAI code and functionality.

EAI is a business issue, not a technical one. Rapidly evolving markets, new
business opportunities and changing customer expectations are driving businesses
to abandon their old stovepipe approaches to IT systems. The creation of new
customer- and supplier-facing systems requires an integrated view of the business,
which can be a real challenge for IT shops with isolated legacy systems. One
helpful tool for understanding EAI is a framework that will help you comprehend
the breadth of EAI, the type of integration you need to do and the business
value of your approach.

Let's begin by defining EAI. That alone is a challenge. Many confuse EAI with
generic middleware. There is some overlap, but the terms are not synonymous.
Let me share two possible definitions with you:

"EAI solutions are software products that completely or partially automate
the process of enabling custom-built and/or packaged business applications to
exchange business-level information in formats and contexts that each understands."
— Ovum, Enterprise Application Integration: Making the Right Connections

"It is essential to keep in mind that it is the chaining together of discrete
transactions, in the form of a business process, from one application to the
next that constitutes EAI." — Cherry Tree & Co., Extended Enterprise Applications

....Note that EAI is not just shipping data back and forth. EAI is also about
processes, the flow of information and the rules that govern that flow. This
is not to say that just exchanging information between systems is not EAI. The
definitions help us understand that EAI is broader in scope and more inclusive.
Cherry Tree & Co. defined an EAI hierarchy that provides an interesting framework
for understanding these definitions. I'll give you my own version of their hierarchy.

The hierarchy has four levels: Transportation, Data Transformation, Business
Rules and Business Processes. At the lowest level of EAI is the Transportation
layer, which is data exchange. This is where messaging and much of what people
think of as middleware fits in. In fact, GartnerGroup defines five application
communication models — Conversational, Request/ Reply, Message Passing, Messaging-and-Queuing,
and Publish-and-Subscribe — that fit in this layer.

Conversational communication is seen as programs interacting with one another
directly. This is usually done through low-level APIs. Request/Reply is similar
to Conversational communication, but is based on a single request and reply.
This may be done with an RPC mechanism. Message Passing is a simple one-way
communication. It is usually unblocked, which means the sending program does
not need to wait for a response. This powerful capability allows systems to
work together asynchronously. Messaging-and-Queuing is a store-and-forward messaging
model with a middle stage. Messages are not sent directly from one system to
the next. Instead, messages go to an intermediate queue that stores them until
they are requested by another system. Publish-and-Subscribe is an informed method
of routing data to interested systems. Essentially, systems subscribe to information
provided by another system. When a system provides new information, that information
is sent to all systems that have subscribed. This approach has the great benefit
of minimizing the number of connections between systems.

Many companies attempt to integrate systems only with the Conversational or
Request/Reply models. After a few of these style integrations have been developed,
your system architecture begins to resemble a plate of spaghetti. The other
models allow more flexibility and robustness, but require additional middleware
to support them. It is in this layer that you find many EAI and middleware vendors.

The next layer up in the EAI hierarchy is Data Transformation. This layer
is often found where architectures integrate systems at the data level. Data
is moved from one database to another, and usually transformed in the process.
This is what database vendors are talking about when they describe their products
as EAI solutions. Other tools, such as those used to scrub data and populate
data warehouses, also fit in this category.

The eXtensible Markup Language (XML) and its associated standards could play
an important role at this level. For example, XSLT-based translations can simplify
the transformation of data formats between systems. XML can also play a role
in semantic and marshaling format transformations. The challenge will be to
control how XML is used. XML could lead to greater data complexity as businesses
struggle to adopt common standards for information interchange.

The third layer in the EAI hierarchy is Business Rules. This layer is where
most full-blown EAI products reside. Essentially, the EAI tools in this layer
use business rules to control what happens in the two lower layers. For example,
an EAI tool such as Mercator will work with a messaging system such as MQSeries
to route and transform information flowing from one system to another. Mercator
may perform the data transformation itself based on defined business rules,
or it may leave that to other middleware. It may also route information based
on its publish-and-subscribe capabilities.

The level of sophistication and control you have with these tools varies from
product to product. Many tools claim they operate in this layer, but matching
their capabilities to the EAI hierarchy helps you understand where they really
belong. Many true EAI tools have capabilities that overlap layers. GartnerGroup
did four case studies with companies using this type of EAI tool. On average,
using this level tool reduced development costs by 33% and maintenance costs
by 66%, a dramatic improvement.

One weakness many of these tools have is their inability to control the flow
of a single piece of information across multiple systems. For example, they
will do a great job of sending information from System 1 to the next system
or systems in line (System 2). What they often don't do is control the flow
of that transformed information from System 2 to System 3 in the same context.
For most EAI systems, each flow is a discrete transaction. They can't see the
entire context of a business process and all of the systems information must
pass through before it reaches its final destination.

This brings us to the fourth and final layer: Business Processes. This layer
of EAI provides for the management of complete business processes. It also covers
the last missing piece of system integration: enterprise workflow. This is the
layer that can add the greatest business value. Tools in this layer will allow
you to reshape and mold processes to match changing business needs.

....Imagine that you've used component technology to create clean interfaces
to your legacy systems, effectively turning them into reusable components. Imagine
that you've also developed your new E-business systems as reusable components.
Enterprise workflow will allow you to assemble and reassemble these components
as needed to create new processes. As you move up the layers, from Transportation
to Business Processes, you improve the flexibility and robustness of your business
processes. You reduce the complexity you need to embed in any given application,
reduce intersystem complexity and shorten development time.

Barriers to EAI adoption

So why doesn't everybody move in this direction? Eventually they will, or they
will be overtaken by their competition. But there are factors that can make
it difficult to adopt EAI. The first is the initial cost of tools. As you move
up the layers, initial costs increase. It is true that each step up can add
significant business value, but many companies find it difficult to swallow
the initial $350,000 to $1,000,000 investment needed in the higher layers.

Secondly, new tools require new skill sets. They can also require changes in
a developer's architectural perspective. Some developers are more comfortable
writing code at the lowest layer and don't want to be bothered learning new
tools. This is why so many companies end up with spaghetti architecture as they
integrate their systems. Training and an understanding of true business value
can help address these problems.

While many tools claim to be the answer to Enterprise Application Integration,
no one tool meets all needs. The EAI hierarchy serves as a framework for understanding
levels of application integration, tool appropriateness and EAI business value.
Try using this framework the next time you evaluate an EAI project or tool.

About the Author

John D. Williams is a contributor to Application Development Trends. He is president of Blue Mountain Commerce, a Cary, N.C.-based consulting firm specializing in enterprise, domain and application architectures. He can be reached via e-mail at jwilliams@bluemountaincommerce.com.