This chapter is from the book

1.1 Distributed Information Systems Everywhere

Typical examples of distributed computing and information systems are systems
that automate the operations of commercial enterprises such as banking and
financial transaction processing systems, warehousing systems, and automated
factories. The Internet has promoted and speeded the growth of distributed
information processing beyond the single enterprise, across the boundaries
between enterprises. Information is shared between di'erent enterprises and
provides the foundation for trading partnerships and the automation of business
collaborations.

Figure 1.1 shows a multi-enterprise financial trading system. These systems
are distributed over various networks worldwide and often use the Internet as
one of the networks. When viewed at a macro level, the various enterprises and
organizations are simply components of the system. Each of them has its own
internal information processing system. The figure shows these components,
including the stock market information system, brokerage houses and online
customers (or more accurately, their workstations), the Federal Reserve,
investment banks, and the networks through which all these components
communicate. Messages (or "events") flow across networks between these
enterprises. They react to the events they receive and issue new events that are
sent to other components. The system is "event driven"it
lives or dies based upon the messages flowing across its IT networks. This is a
rather large system with high volumes of messages flowing through its networks.
For example, in 2001, 5,000 or 10,000 messages per second flowed through a
single large brokerage house's information technology layer. Soon, that
number will be higher.

Figure 1.1
A system of distributed communicating enterprises (*photo by Wonderlife USA Corporation)

Financial trading systems are but one example of distributed IT systems.
Generally speaking, the business operations of any global coporation are
supported by a widely distributed, message-based computer system. Figure 1.1, by
the way, shows tools for complex event processing added to the distributed
systemwe will explain this later. In fact, that's what this book is
about.

Government and military information systems are also distributed systems.
Figure 1.2 depicts a typical military command-and-control system that links
command centers, intelligence-gathering operations, and battle-field units of
all servicesa so-called C4I (command, control, communications,
computers, and intelligence) system. At first look, a military system may seem
to be entirely di'erent from a commercial one. Certainly, the military
goals and operational conditions are very di'erent, and many of the types
of component objects in military systems are di'erent from commercial
systems. But military systems also contain a lot of commercial components, such
as operating systems and databases, and often parts of their IT layer use the
same publicly available networks, such as the Internet.

Figure 1.2
A command-and-control system monitored by an event processing network

In fact, all three kinds of systems have a lot in common. In all casescommercial,
government, and militarythe underlying architectural structure is the
same: a distributed information system consisting of a widely dispersed set
of several thousands or hundreds of thousands of application programs (or component
objects, as they are often called) communicating with one another by means
of messages transmitted over an IT layer containing various kinds of media.

All these systems have come to be collectively called "enterprise
systems." They all have a common basic problem. Their activities are
driven by the events flowing through their IT layers. And they produce zillions
of events per hour or day. But there is no technology that enables us to view
those events and activities that are going on inside these systems in ways that
we humans can understand. To be sure, given the primitive tools we have at the
moment, we can see the events. But making sense of them is the
problem!

Enterprises invest as best they can in tools to monitor events in the basic
networks that carry information. So we are told, for example, that "the
router in Hong Kong is overloaded." We have to figure out that the router
problem may be holding up completion of an important trading agreement between
our New York o=ce and our partner's Tokyo o=ce. We need to be able to
answer questions about events that are not simply low-level network activities,
but are high-level activities related to what we are using the systems to
doso-called business-level, or strategic-level, events. We need answers to
questions like these:

"What caused our trading system to sell automobiles (a
business-level event) to a customer in Texas?" The answer could involve
several other trading transactions.

"Is the system, at this moment, under a denial-of-service
attack?" The answer requires real-time recognition of complex patterns of
events, the patterns that indicate such attacks.

"What is causing the system to fail to execute this trading
agreement?" The answer could be a missing set of earlier business-level
events, like failures of several suppliers' systems to respond within time
limits.

These questions are about complex events, which are built out of lots
of simpler events. Answering them means that we need to be able to view our
enterprise systems in terms of how we use themnot in terms of how
we build them, which is the state of the art in enterprise monitoring
technology today.