Article

The exchange infrastructure is SAPs new XML-based application integration
platform. Tightly integrated with SAP applications but open to non-SAP
components, it provides a technical infrastructure and delivers business-process
and integration knowledge  in the form of SAPs predefined
integration scenarios  to the customer. Whats more, the exchange
infrastructure offers an integrated toolset to help you build your own
integration scenarios by defining the appropriate messaging interfaces,
mappings, and routing rules needed to connect application components or
to create collaborative processes involving business partners.

In this overview, we will take a look at
all the elements that make up the exchange infrastructure, and how they
connect SAP components with each other, as well as with non-SAP components.
Although the first release will not provide all the functionality discussed
here, this introduction is designed to help as you plan for the short-
and long-term future of your current system landscape. With a blueprint
of the exchange infrastructure in mind, you can start to evaluate how
to make a gradual, step-by-step transformation to a more open, flexible,
and integrated system environment.

XML Data Exchange  Flexible and Open Communication

One of the key principles of the exchange infrastructure is that data
is exchanged between application systems in the form of XML messages.
XML provides a whole new set of standards and technical building blocks
that  although they do not solve all the issues of semantic integration
 provide a huge step forward in terms of interoperability.

XML is not only used for meta data, it
is also the syntax for everything that is exchanged between heterogeneous
systems. After all, when it comes to collaborating with business partners
directly or via electronic marketplaces, we know that asynchronous and
guaranteed messaging is the only option for critical business transactions.
So why not make the external and internal integration look the same, and
benefit from a homogeneous integration infrastructure?

With XML, you have a flexible basic format
for transporting and transforming information. Message formats can be
based on open XML standards for business documents, or can be custom designed.
Using XML-based mapping, it is possible to transform messages to and from
a common XML format or directly from inbound into outbound format.1

This approach is similar to ALE and IDoc
communication in that the asynchronous XML messaging allows for the decoupling
of applications. Since the exchange infrastructure is designed to use
XML and open Internet standards to describe all the necessary meta data,
this data can be kept in one central location, which shifts integration-relevant
information away from the application. This makes for a much more flexible
IT landscape, and one that can be more readily adjusted to evolving software
or business conditions. Individual systems can be added, maintained, or
taken out of the landscape, while the rest runs as usual.

Taking the First Step: Incorporating the Exchange Infrastructure Into
Your System Landscape

Imagine a simple IT landscape. Youve got:

An SAP R/3 4.0B system

An SAP R/3 4.6C system

A third-party application system

A legacy application

Even in this simple example, you would find various methods of data exchange:

Between the legacy and the third-party system, you find any number
of custom-programmed file exchanges.

Between the third-party and SAP R/3 4.0B system, you find some BAPI
calls.

Between the R/3 systems, you find ALE communication.

What can the exchange infrastructure do
for this system landscape? Figure 1 illustrates the possibilities.
For starters, with the exchange infrastructure you can take the existing,
file-based data exchange between the legacy and the third-party systems
and re-route it through the Integration Server, using a more modern XML-based
interface to communicate with the third-party application component. This
has several benefits:

Knowledge about the integration of these two systems is kept within
the exchange infrastructure, making it more manageable and easier to
change in the future.

The data exchange between these systems becomes monitorable 
so you can tell which data files have been picked up and processed and
when.

Later, the same data, once it is converted into an XML message, can
be reused to communicate with other connected systems in this landscape.
For example, via mapping, it can be made available to the R/3 system
in the form of an IDoc.

Figure 1

A Simple Landscape Incorporating the Exchange Infrastructure

Many customers see this kind of scenario
as a very important step on the way from their existing IT landscape to
a more message-based, process-oriented environment with consistent data
throughout.

Of course, the real value does not lie
in the single interface integration described here, but in making it part
of an overall integrated landscape, where other receivers can make use
of the same data and entirely new, collaborative business processes become
possible.

Lets take a look now at the technology
that makes the exchange infrastructure work.

The Path to the Exchange Infrastructure Is Simple and Painless

Introducing the exchange infrastructure into a system landscape,
as we did in this example, is simple and painless. We didnt
even need to touch the BAPI and ALE communication between the R/3
systems.

Remember that were advising
an evolutionary approach to adopting this technology. When the time
is right, you introduce XML-enabled interfaces, utilizing the exchange
infrastructure as a platform for communication between the connected
application systems.

Understanding the Elements and Workings of the Exchange Infrastructure

The exchange infrastructure is made up of the following major components,
shown in Figure 2:

The Integration Repository, which stores design-time integration
information  such as interface descriptions or business scenario
descriptions  delivered by SAP or entered by you.

The Integration Directory, which stores only the integration
information that youve configured for your own IT landscape.

The Integration Server, the runtime environment for performing
all tasks involved in controlling business processes and exchanging
data between connected systems.

The Integration Repository and Integration Directory are used to store
what we call shared collaboration knowledge: all the meta data
that describes the integration of application components and business
processes. Both the Repository and the Directory are built on Java technology
and use XML standards for all object descriptions. They not only allow
the integration of SAP components, they can also hold information about
other systems interfaces and services, and can model business processes
across heterogeneous landscapes. While the Repository and Directory are
closely linked, they have very different roles.

The Integration Repository holds
design-time information. As such, it serves a dual purpose: it
is part of the development environment, and it also delivers a vast amount
of accumulated design-time information that the customer can use as a
resource in the configuration process.

The Integration Repository tools function
as a starting point for the design of interfaces, XML message schemas,
and business processes. SAP performs this kind of development and packages
this integration knowledge in the Repository, so that it is available
as a basis for configuring the actual integration landscape. It will also
be possible for customers and partners to add to the Repository with their
own applications developed on the SAP Web Application Server and using
the exchange infrastructure.

If the Repository is the starting point
for integration scenarios, the Integration Directory reflects the
finished product  the integration that is actually in place in
a particular landscape. The Integration Directory holds the configuration
information for the customers integration landscape: which application
components are actually installed in the IT landscape, which business
processes are actually configured, which interfaces on which systems are
actually called, and so on. When XML messages actually flow from component
to component, all this data is made available to the runtime environment
 the Integration Server  to control the execution of all relevant
integration tasks.

Information in the Integration Directory
is linked to and references the corresponding information in the Integration
Repository. As a result, not only is configuration-time knowledge kept
in synch with design-time knowledge, but the information is also stored
centrally, rather than scattered across all the various participating
systems and applications.

The Integration Server: The Communications Platform of the Exchange
Infrastructure

The Integration Server is made up of a number of components. The main
component is the Integration Engine, which handles processing,
mapping, and routing services. A Business Process Engine will also
be included in the Integration Server to control complex and collaborative
business processes. Additional Integration Services may be included
for particular integration purposes, such as special value-mapping or
master data integration. Adapters enable the Integration Engine
to exchange data with different application components.

In the future, as new SAP applications
are developed based on the latest mySAP Technology  such as the
recently announced Supplier Relationship Management (SRM) component 
they will be able to communicate with the Integration Server without requiring
additional adapters. Each new application will contain its own Integration
Engine for handling the local messaging and integration tasks, and will
be able to communicate with other applications, or directly with an Integration
Server (see the sidebar below).

While all parts of the exchange infrastructure
run on top of the SAP Web Application Server, participating applications
and special adapters may run on any platform that supports basic Internet
protocols, such as XML, HTTP, SOAP, and so on. Since all Integration Engines
work off the same centrally stored and managed integration information,
the term shared collaboration knowledge should now make perfect
sense: knowledge about integration is shared over a distributed execution
environment.

New SAP Components Will Be Designed for Direct Integration into
Your SAP Landscape  No Adapters Needed

How would the exchange infrastructure
integrate a newly developed mySAP.com
component into your existing SAP landscape?
Suppose you
have a new mySAP.com component  such as the Supplier Relationship
Management (SRM) solution  that needs to communicate with
an existing SAP R/3 4.6C component (see Figure 3). How would
that scenario be supported?

In this case, the SRM component needs
to send an XML message asynchronously to an RFC interface in the
SAP R/3 component. As you will see
in many future SAP solutions, an Integration Engine is already incorporated
into the SRM component.2 This Integration
Engine takes the XML message, makes it persistent (to support guaranteed
delivery), then forwards the message via HTTP to the Integration
Server, which acts as the communications hub.

Here, the Integration Engine receives
the message from the HTTP handler, makes the message persistent
in a queue on the Integration Server, and then executes the logical
routing and mapping steps by calling the appropriate services in
the integration layer.3

Finally, after the Integration Server
determines the physical address and protocol characteristics of
the receiving R/3 system, the XML message is passed to an RFC adapter.
This adapter then executes the asynchronous call with the RFC message
data to the R/3 system via the tRFC protocol.

Figure 3

Integrating a New mySAP.com Component into Your
Landscape

The Integration Engine: Processing and Delivery of the XML Message

The Integration Engine is part of the Integration Server, and helps to
exchange all messages between the various components connected within
the system landscape. It has two layers: Messaging & Transport for
basic message handling, and Integration for routing and mapping.

Once an XML message is received by the
Integration Engine, it is handled by the Messaging & Transport Layer.
To this end, a pipeline in the Integration Engine executes integration
services, such as logical routing or mapping, as well as more technical
services, such as message persistence, queuing, and logging provided by
the messaging layer itself. In this layer, the Integration Engine provides
queuing functions for the physical decoupling of application components
and goes beyond standard Internet protocols to guarantee the exactly-once
delivery of messages. For example, messages are passed between Integration
Engines as SOAP messages with specific header extension fields to provide
reliable messaging.

Once the Integration Layer receives a message
(with information on the sender and the message interface), the logical
routing service determines the correct receivers and required interfaces
by evaluating the corresponding routing rules, whether those rules are
XPath expressions or Java code. The configuration of routing rules allows
the Integration Engine to use the message content to determine the receivers
for messages in a publish/ subscribe model. Routing rules help relieve
the sending application from the need to know all the details about potential
receivers and each of the message formats that the receivers will understand.

The mapping service then determines the
required transformations, depending on the sender and receiver interfaces.
After retrieving the required mapping from the Integration Directory,
the mapping service can execute XSLT mappings or Java code (or a combination
of the two in the right sequence) to the business content of the sender
message. An XSLT processor and the Java execution environment are contained
in the underlying SAP Web Application Server. Both XSLT and Java have
the benefit of being a kind of universal language that can be understood
and executed on multiple runtime environments. This is the basis for the
evolution of the technical infrastructure itself.

Adapters Link Integration Engines to R/3 and Third-Party Systems

Adapters enable the Integration Engine to talk to different application
components that are not able to speak to web service protocols natively.
However, adapters address more than just technical connectivity 
they also address semantic issues.

As illustrated back in Figure 2, third-party
adapters link an Integration Engine to third-party systems like Baan,
Oracle, and PeopleSoft, as well as to legacy systems. These adapters translate
XML and HTTP-based messages into the (often very specific) protocols and
formats of a given application system.

To integrate existing SAP components like
R/3 Release 3.x and 4.x, the IDoc or RFC adapters establish connectivity
between the exchange infrastructure and SAP systems. These adapters translate
XML and HTTP-based documents into IDocs and RFCs. This ensures that companies
can still seamlessly connect to their existing SAP infrastructure while
leveraging the new exchange-based integration infrastructure. (For SAP
systems based on the Web Application Server 6.20 and up, SAP supports
direct systems connectivity without the need for additional adapters.)

A number of generic adapters can be used
either to integrate application or technical systems by exchanging files
or accessing databases, or to integrate other messaging systems such as
IBMs MQSeries via the JMS API. (Note that such an external queuing
system is not needed for queuing messages within the Integration Server
itself, but can complement the server as a communication partner for the
exchange of messages.)

Built-In Monitoring, Tracing, and Error Handling

The Messaging & Transport Layer is built on top of the ABAP stack
within the SAP Web Application Server. This provides SAPs proven
system infrastructure technology as a basis for features such as message
persistence and queuing, and also allows for integration with the standard
SAP monitoring infrastructure.

Depending on the configuration of the Integration
Engine, XML messages can be logged at all points in the processing sequence,
and can then be searched, displayed, and compared in a monitoring environment.
The sending system can also request that the Integration Engine provide
detailed tracing information as a message attachment. The Integration
Server itself, along with important aspects such as queue status, can
be monitored via CCMS.

The Integration Engine is designed to handle
different error conditions in appropriate ways.
For example, a temporary inability to reach
a target system will be handled by a retry
mechanism in the messaging layer. Certain error
situations are detected automatically, such
as the inability to deliver messages due to
missing configuration information. The Integration
Engine reacts by putting such messages in special
error queues and notifying the administrator.
For other kinds of errors (for example, application-level
errors occurring as a result of an asynchronous
messaging process), the Integration Engine
logs fault messages returned from the application
component and makes them available for follow-up
processing.

Lets revisit our simple system landscape to get a better
picture of how, with an XML message, our systems can be integrated.
Remember that in this landscape, data from a legacy system was written
to a file, routed through the exchange infrastructure, and sent
to one of our SAP systems. In Figure 4, you can see a blueprint
of an Integration Server containing only those adapters necessary
for this particular scenario, as a file is transformed to an XML
message, and then to an IDoc for use in an R/3 4.6C system.5

Figure
4

Blueprint of the Integration Server

For purposes of illustration, we
will follow this XML message on its way through the exchange infrastructure,
room by room so to speak.

Before we begin, though, lets
look at how an XML message is created in the first place. The legacy
system, as part of its daily work, has just written data to a file
on the file system. The data is plain, comma-delimited ASCII data.
The file adapter in our Integration Server is configured to monitor
that system, so it picks up the file, extracts the data from it,
and an XML message is born.

If you think about the anatomy
of our XML message (see Figure 5), its head contains
the essential information for processing the message within the
Integration Server: the message type, the sender (the legacy system),
and so on. It also indicates that delivery to the target system
should be asynchronous and guaranteed exactly once (quality of service).
The body of our rough XML message looks very simple, actually 
it is just a collection of tagged items, organized almost like a
table of values.

Figure 5

A Rough Sketch of an XML Message

Now were ready to begin the
Grand Tour. In the first floor of the Integration Server, we follow
the newborn message on its way into the Integration Engines
main room, the Runtime Engine. This is the central point from which
all other rooms branch off. On its way to each part of the Integration
Engine, the XML message will keep returning here.

Since the delivery of the message
to the target system needs to be guaranteed by the Integration Server,
we follow the message next to the basement, where processing
queues and other integration runtime data are stored in the database.
(The system keeps an image of the message here in case of processing
errors or system failure.)

Now the message returns to the Runtime
Engine room, and the actual processing begins. In the Logical Routing
room, the exchange infrastructure determines where to send the message
based on the messages header information, and perhaps on its
body as well (for content-based routing). To make these routing
decisions, the Integration Engine quickly accesses the routing model
data in the Integration Directory in the basement.6
The receiver information is added to the messages header for
later reference.

Once the Integration Engine has identified
all the places the message needs to go  in this case, to just
one SAP system  it then determines how to get it there. In
the Technical Routing room, this information is retrieved from the
Integration Directory: the receiving system is an SAP R/3 4.6C system,
and the receiving interface is an IDoc interface. This is added
to the header as well.7

Next stop is the Mapping room, where
parts of the messages body are taken off and reattached elsewhere
(ouch!) as part of structure mapping (done with XSLT or Java). This
extensive cosmetic surgery makes the XML message look
like an IDoc; however, it still hasnt gone through its full
transformation.

Our message now returns to the Runtime
Engine room one last time, and here it is pushed into the IDoc Adapter
room. The IDoc Adapter takes the data, produces a regular IDoc,
and sends it to the connected R/3 system via a regular tRFC.

The record of our XML message remains
in the database, at least until its time for archiving or
the data is extracted for analysis.

As we were taking this tour through
the exchange infrastructure, you may have noticed that we didnt
enter the Business Process Engine and Additional Integration Services
rooms. This floor is still under construction, but watch for new
functions to be added on, such as value-mapping tables from a master
data management solution, message correlation, and event management.

Business Process Engine Brings State Management to Messaging

Future additions to the Integration Server will include the Business
Process Engine, which will introduce robust state management. Whereas
the messaging layer works in a stateless manner, the Business Process
Engine will be responsible for tracking multiple messages to provide the
state management required for business processes. In other words, it will
manage the correlation between messages, and remember the
status of the exchanged messages  what is done, and what is left
to do  based on the messages sequence and content. (Currently
WebFlow technology handles these processes, but the Business Process Engine
will expand these functions. For more information, see the article Bursting
the Boundaries of Your Current Business Processes by Alan Rickayzen
in this issue of SAP Insider.)

In this role, the Business Process Engine
can use the messaging layer to trigger the execution of process steps
on other component systems, suspend the process, and resume execution
when response messages are received.

Future System Landscapes with the Exchange Infrastructure

Now that weve laid out some of the possibilities for your current
landscape, lets fast forward to what a more complex system landscape
may look like in the future, making full use of the integration capabilities
of the exchange infrastructure.

Figure 6 illustrates three sub-landscapes:
one for intra-enterprise (internal) process integration (which we will
call Production, for our purposes), one for running collaborative business
scenarios with business partners in a private exchange setting, and one
for testing purposes. Youll notice the Integration Repository is
now shared by all three landscapes as a common development and meta data
repository. Multiple Integration Servers and Integration Directories are
set up to separate the individual landscapes from each other for security,
load balancing, or software logistical reasons.

Figure 6

Looking Ahead to the Next Steps for the System Landscape

Existing SAP and third-party components
can be connected to a separately deployed Integration Server, which, for
these components, acts as a communications hub. New components deployed
on the SAP Web Application Server can use their built-in Integration Engines
to communicate directly with each other, or through an Integration Server
with other components  all depending on the configuration of the
individual Integration Engines. It is important to point out again that
while a distributed execution environment is possible, all Integration
Engines use the same centrally stored collaboration knowledge.

Conclusion

Are you thinking about what your companys IT landscape will look
like in the next year, in five years, and into the future? Are you planning
to set up new, collaborative business processes, or integrate systems
previously not connected? Do you foresee the consolidation of functionality,
now spread over numerous legacy systems, by introducing new application
components?

Then consider how your organization might
benefit from the exchange infrastructure, used
in a step-by-step process to arrive at a homogeneous
infrastructure for integrating a heterogeneous
world of application components and business
processes.

1
Why would you choose anything but direct inbound-outbound transformation?
Not only is a common XML format a more standards-based approach, it can
actually reduce the total number of
necessary mappings. Suppose you have 2 applications
sending sales order documents, and 3 applications
receiving sales order documents, each one with
a different sales order interface. If you individually
map each sending interface to each receiving
interface, you get a total of 2*3=6 mappings.
On the other hand, if you introduce a common XML
sales order format in the middle, and map each
sending and receiving interface to that, you
only get 5 total mappings. Extrapolate this
to 10 sending and receiving applications each,
and you can imagine the benefits!

2 SRM
contains a built-in pipeline that
takes the XML message from the application
proxy, which was generated from the interface
definition in the Integration Repository
during SRM application development.

3 Depending
on the configuration of the pipeline, additional
logging steps may be included while processing
the message.

4 Although
you are probably aware that XML processing
does produce a certain overhead, I want to
reassure you that it will not take a whole
day to process a single message!

5 For
simplicity, the Integration Repository and
Directory in this example have been included
in the Integration Server, although a distributed
installation will be possible as well.

6 We
are not really accessing the Integration
Directory directly, but instead the routing
model data, which has been cached in the
Integration Engines runtime data store.

7 However,
had the message been destined for three receivers,
there would have been copies made in the
Integration Server, and each one would be
processing separately.

Sven
Leukert joined SAP in 1998, after studying mathematics in Berlin and receiving
a Ph.D. in mathematics from the University of North Carolina, Chapel Hill.
He worked in product management for CRM Middleware until 2000, and has
been involved in the development of the exchange infrastructure from its
beginnings. He is currently a product manager for the exchange infrastructure,
and is responsible for selection of pilot customers.

More from SAPinsider

As digital technologies have expanded their reach, the task of ensuring secure authentication and identity management within business processes has become more complex. Employees are no longer the only users...