Architectural Patterns for Distributed Computing

Summary: As the technologies supporting
design, development, and deployment of distributed applications continue to
advance, it is more important than ever for architects and developers to
establish stable libraries of architectural patterns based on recurrent
experiences developing such distributed solutions. This article will explore
patterns harvested from customer engagements—mapping real-world scenarios to
patterns and then describing how said patterns were implemented on current
technologies. Our analysis also includes a discussion of the forces leading to
the application of a particular pattern and the resulting context that must be
considered.

Contents

Introduction

Distributed systems architecture has rapidly evolved by applying
and combining different architecture patterns and styles. In recent years,
decoupling interfaces and implementation, scalable hosting models, and service
orientation were the prominent tenets of building distributed systems.
Distributed systems have grown from independent enterprise applications to
Internet-connected networks of managed services, hosted on-premise and/or
clouds.

We will discuss patterns harvested from different consulting
engagements and examine how these patterns were implemented on current
technology offerings from Microsoft. Building distributed systems involves
three core capabilities, based on the following roles:

Implementation of applications/services.

Consumption of services from within and external
to organizations.

Administration of services or composite
services.

Implementation of Services and Business Processes

Let’s start with the implementation capabilities needed to build
and expose services. Implementation capabilities describe an organization’s
methods to implement effective best practices, identify and apply patterns, and
provide services.

Service Design

Five years ago, the Patterns & Practices (P&P) team was
asked to create a reference application for developing distributed applications
to accompany the release of the second version of .NET Framework. To design
this reference application, P&P spent much time talking with customers and
consultants in the field to understand the challenges they encountered when
designing, developing, and deploying applications. As a result of these
interactions, P&P identified recurring patterns relevant to SOA from
narrative-based pattern guides culminating with the Web Service Software
Factory.

Patterns. An early promise of services was support for interoperability
within a heterogeneous environment, but given the wide range of technologies
and emerging nature of many of the WS* standards, it was critical for the
service’s business logic to be completely independent of the service’s
interface, allowing the bindings and policies associated with the service to
evolve separately. Furthermore, the actual technology used to implement the
services could also minimize the impact on business logic. The Service
Interface pattern described a means of designing services to cater to these
forces.

In addition to separating the design of the service interface
from the service’s implementation, cross-cutting concerns such as
authentication, authorization, message validation, and auditing were
implemented using a pipeline similar to the Pipes and Filters pattern. This
pipeline allowed these cross-cutting concerns to be specified declaratively
independent of the service’s business logic. This pattern is used within the
WSSF to ensure that cross-cutting concerns such as validation are configured
declaratively.

A core tenant of SOA is the concept that a service’s boundaries
are explicit; interactions are performed by using well-defined messages.
Furthermore, the service must ensure that such messages provided as input are
validated because incoming data may be malformed and transmitted for malicious
reasons. The service must also ensure that erroneous behaviors occurring within
the service related to the internal implementation of the service are not
accidentally leaked. The Message Validator pattern describes several approaches
that can validate incoming messages prior to execution of business logic. The
WSSF actually uses the Enterprise Library Validation Block to support these
requirements. A second pattern, called the Exception Shielding pattern,
describes precautions that a service author can take to ensure that exceptions
are caught and sanitized prior to returning a fault to a client, while still
ensuring that sufficient information is logged for administrators to troubleshoot.

Figure 1 illustrates this patterns interaction.

Figure 1. Exception Shielding pattern

Finally, when it came time to deploy these early services to a
production environment one challenge that many customers faced was that the
operations folks would not allow the clients to talk directly to the Web
service. For customers with external clients, it was against company policy to
have an external application interact directly with an application residing on
an internal network. The solution was to deploy a Perimeter Service Router into
the perimeter network, allowing it to take responsibility for directing
incoming messages to appropriate internal resources.

Figure 2 illustrates this scenario.

Figure 2. Perimeter Service Router pattern

Process Design

The goal of any IT system is to implement the logic behind an
organization’s business processes. As organizations automate more and more
processes, the IT department’s perceived value grows, as well as its
responsibilities on the business-process realization itself.

This is particularly true in industries where the products are
materialized as IT systems, such as areas of commerce and trading, banking, and
insurance. An IT department’s effective implementation of the business process
can lead to predominant market positions and impressive returns. In turn, IT
inertias and the inability to execute quickly can lead to business disasters.

Cost and time important considerations when designing and
implementing business processes is the need to optimize cost and time. To
reduce costs, reuse effective patterns, software factories, and
agile/test-driven software methodologies. Noncompliance with specifications and
changing business processes and policies during the course of the
implementation contribute to delays.

When dealing with reuse effectiveness, the graphical composition
of complex processes from small reusable building blocks and the augmentation
of those technologies with domain-specific extensions showed greater value. A
good system will allow for process templating—for example, the ability to
create a macro process template that derives a number of processes that follow
the same template. The usage of graphical composition tools allows non
developer users to review and change processes. In order to run a process, you
need a hosting environment, as well as the ability to locate and read the
process definition and efficiently execute it. In addition, environments should
provide reliability, fault tolerance, on-demand migration to different
environments, and transactionality.

Scenario Description. We are involved with a banking automation
project in Europe. This bank is active in the retail market with a large
diffusion of branches across the territory, and is gaining a lot of traction
from its customer-directed financial products.

Bank management has opted for a strategy that is based on four
pillars:

Maintain customer contact, but steer the branch
from a mere teller office to a financial services shop.

Advanced workflow patterns—Wizard-based tools
that drive contract-first development, inclusion of transparent processes or
intermediaries, context-based process selection, enabling stateful
interactions, and externalization of business rules. Some of these patterns are
not formalized elsewhere.

Business-activity monitoring—Monitoring the current
state of a business activity.

These patterns force you to build service compositions with the
flexibility to dynamically change the activities through an appropriate service
implementation based on business rules.

Technologies Realizing Patterns. Microsoft offers two
technology solutions to address business-process development and hosting:
BizTalk server and Windows Workflow (WF). BizTalk offers a platform and tools
to connect with services inside and outside of organizations, including a large
number of adapters to legacy systems. WF is the programming model, engine, and
tools for quickly building workflow-enabled applications. In addition,
Microsoft consulting developed a wizard-driven toolkit and runtime, Distributed
Connectivity Service (DCS), to provide guidance on developing and hosting
business processes that are based on the aforementioned patterns. DCS provides
service-management capabilities such as service location transparency, dynamic
clients, context-driven business-process selection, scalable hosting of
workflows, and a security token service for exposing business processes as
services. Depending on the enterprise scenario and hosting requirements, you
could select BizTalk, DCS, or a combination.

Table 1 lists the patterns category, along with the problem
statements and associated technology implementations.

Table 1. Business-process workflow-design patterns category

Patterns category

Problem

Solution

Process-driven development

How to develop business process conforming to the well-known
workflow patterns; how to adapt quickly to process-change requirements

How to intercept service calls transparently, modifying their
behavior via stateful interceptors able to correlate multiple operations—for
example, to apply new marketing-driven, customer state-specific behaviors

Deploying Service

Deploying services to a production environment is complicated.
Operational staffs need to decide how the service should be exposed, who has
access to the service, which protocols should be supported, and how to version
a service. In some cases it is against company policy to have an external
application interact directly with an application residing on an internal
network. Some customers restrict service access to a specific set of
credentials. Organizations would like to expose multiple versions of the
service for old and newer clients. Services may use legacy protocols that clients
don’t support.

Consumption of Services

The consumption of services from within the enterprise (as well
as those external to it) requires business enablement and platform
capabilities. These capabilities help enterprises to effectively adopt and
promote the use of services by providing a foundation to support and enhance
the consumption of enterprise services by others.

Enterprise Service Bus. The term Enterprise Service Bus (ESB)
is widely used in the context of implementing an infrastructure for enabling a
service-oriented architecture (SOA). An ESB product supports a variety of
patterns, hence by breaking various ESB’s down into their constituent patterns,
you get a real sense of the capabilities implemented within each solution.

Scenario Description. We worked with a major insurance provider
that used the BizTalk server as a platform choice for the integration of legacy
systems. Recently, this company became a provider of auto claim services.
Receiving an auto claim from claim office, creating the claim process, and
sending the claim to an independent adjuster for damage inspection and
receiving a repair estimate for approval, parts procurement, and further
repairs is a common scenario. These imply configurable policy-based service
mediation and composition of enterprise and third-party services, thereby
coordinating the execution of distributed business processes.

Patterns. You can think of an ESB as a collection of
architectural patterns based on traditional enterprise application integration
(EAI) patterns, message-oriented middleware, Web services, interoperability,
host system integration, and interoperability with service registries and
repositories.

Applying message-routing patterns forces you to build service
compositions with flexibility in a dynamically evolving enterprise:

Routing Slip Pattern—Routing a message through
consecutive number of steps when the sequence of steps is not known and may
vary for each message

Recipient List Pattern—Routing a message to a
dynamic list of recipients

Content-Based Router—Routing a message to its
destination, based on message content

Scatter-Gather—Maintaining the overall message
flow when a message must be sent to multiple recipients, where each may send a
reply

Repair and Resubmit—Modifying content of
externalized fault that contains the original message and resubmitting to
resume the processing sequence

Applying message-transformation patterns forces you to use
canonical data models to minimize intra-application dependencies when different
data formats are being used:

Data Transformation—Transforming a message from
one format to another, when the format is semantically equivalent

Content Enricher—Communicating with another
system to include additional information in the message content

Technologies Realizing Patterns. To implement best practices
for building service compositions, developers and architects should clearly
articulate the following:

Identify a baseline architecture that provides a
view of an executable slice through the overall system that is designed to
implement the significant use cases.

Choose integration topology that suits most of
the known requirements.

Incorporate functional scenarios into baseline
architecture by mapping the design into a set of known integration patterns.

Table 2 lists the patterns category along with the respective
problems and solutions.

Table 2. Service-Composition Patterns categories

Patterns category

Problem

Solution

Message-routing patterns

How to define service compositions

How to enforce development best practices for service discovery

Use the Routing Slip pattern for implementing
itinerary-based routing.

Leverage key itinerary abstractions to compose
messaging and orchestrations into a sequence of itinerary steps.

How to force the reuse of complex messaging scenarios
independently from service contracts

Use service registry and categorization
schemas for service discovery.

Define service orchestrations as reusable
components. These components can be incorporated as reusable itinerary steps.

Message-transformation patterns

How to apply data-model transformations dynamically

Define data-model transformations to be
executed as part of itinerary-based routing.

Operations management

How to troubleshoot messaging activities, repair, and resubmit
a message

How to track execution of itinerary-based routing

Apply the Repair and Resubmit pattern by
providing infrastructures with capabilities of externalizing faults to a
centralized store and use management applications for generating alerts,
message editing, and resubmission to a service endpoint when applicable.

Use BizTalk BAM to track message interchanges
and execution history of itinerary steps.

Patterns Addressing Internet Service Bus (ISB) Scenarios

Internet Service Bus (ISB) addresses cross enterprise service
collaboration scenarios. Organizations attempting to implement such scenarios
with a traditional Web services composition/ESB approach are facing challenges
when it comes to spanning business processes across the enterprise boundaries.

In this section, we offer a design pattern, Agent Traveling
Design Pattern, and specifically two of its subsidiary patterns, Itinerary
Pattern and Ticket Forward Pattern, which can help to solve these challenges.
Traveling patterns deal with various aspects of managing the movements of
mobile agents, such as routing and quality of service. Examples of the
Itinerary pattern can be found in various implementations of ESB.

We will demonstrate how with help of MSE and ESBG one can achieve
the desired Itinerary pattern architecture that spans network and geographic
boundaries.

Scenario Description. We are involved with a medical insurance
provider that decided to embark on SOA to modernize its Information technology
while leveraging existing legacy data centers and Web services deployed across
the nation. Its main data center is connected in a cross-star network to
several satellite data centers as well as to the third-party partner’s medical
claims management agencies.

This provider decided to embrace all of its data-center services
and LOB applications into a set of workflow processes, utilizing the Itinerary
pattern, which is dynamically controlled by the business rules, runtime
policies, and SLAs.

Operating in an environment with several
geographically dispersed data centers.

Implementing multiple itinerary workflows, some
of which are nested in each other.

Providing the ability for real-time monitoring
and routing of transactions.

Patterns. For evaluation purposes, we selected the Itinerary-Ticket
pattern for this customer scenario, which is a combination of two traveling
patterns: Itinerary pattern and Ticket pattern. The Itinerary pattern is an
example of a traveling pattern that is concerned with routing among multiple
destinations. An itinerary maintains a list of destinations, defines a routing
scheme, handles special cases such as what to do if a destination does not
exist, and always knows where to go next. Objectifying the itinerary allows you
to save it and reuse it later. The Ticket pattern is an enriched version of a
URL that embodies requirements concerning quality of service, permissions, and
other data. For example, it may include time-out information for dispatching an
agent to a remote data center in our customer scenario. It can also contain the
security and auditing metadata required by the regulations and controlled
(injected) by the execution policies. Thus, instead of naively trying to
dispatch to a disconnected host forever, the agent now has the necessary
information to make reasonable decisions while travelling. An agent is capable
of navigating itself independently to multiple hosts. Specifically, it should
be able to handle exceptions such as unknown hosts while trying to dispatch
itself to new destinations or make a composite tour (for example, return to
destinations it has already visited). It might even need to modify its
itinerary dynamically. Consequently, it is probably preferable to separate the
handling of navigation from the agent’s behavior and message handling, thus
promoting modularity of every part.

We developed an architecture consisting of BizTalk Server, MSE,
and ESB Guidance components. In this case, all three of Microsoft’s SOA
technologies were brought into play to provide the required functionality.

Service Administration

Administration covers systems management, governance, and
operational aspects of distributed systems. Most distributed system
implementations today are referred to as “well-understood and well-behaved.”
These implementations assume that the services being used are in a controlled
environment, usually in a single data center or across a trusted partner
channel. Increasingly, the Internet is being used as the delivery vehicle for
innovative new service offerings such as Web 2.0. Businesses are developing
bundled service offerings to provide value-added services to their customers
over the Internet. These service offerings increasingly rely on the ability to
aggregate multiple composite services from a wide variety of different sources.
The challenges include service-level management, resource management, service
provisioning, and monitoring. This mandates services to follow the principles
of service enablement and supports methods for discovery, provisioning,
monitoring, security, and usage measurements. These management features are
similar to the well-known telecommunication patterns of Fault, Configuration,
Accounting, Performance, and Security (FCAPS).

Scenario Description. A few years back we were involved with a
telecommunication service provider to automate its service delivery platform
infrastructure. This company leveraged Web services capabilities to develop and
expose its core service offerings and reused services from other service
providers such as geo-location providers, ring-tone providers, and credit
verification systems; for example, bundle a ring-tone download service from a
public service provider along with network services and generate billing
information for each successful download. This system is always available and
the provider monitors the usage of the third-party services to validate the
SLA.

Developing and deploying services for
manageability and operations-friendly.

Making services easily discoverable and
composable.

Monitoring the health of individual and composed
services.

Collecting metrics on service usage and
performance.

Enabling automated provisioning of services.

Patterns. This experience helped us discover a series of
patterns that are related to service management. These patterns and best
practices have a deep-rooted background in building telecommunication network
best practices, which is considered the most complex distributed-computing
network with highly distributed and highly reliable systems.

Operations-Friendly Service-Modeling Patterns. It is a best
practice to adopt the principles around knowledge-driven management while
modeling services or composite services:

Exceptions are unavoidable—Always accept
failures and build systems to adapt and provide recovery methods. Also, show
exceptions as needed rather than flooding systems with the same or related
exceptions.

Collecting metrics and logs—Collecting
information regarding failures, performance measures, and usage will help
identity the failure conditions and service-level management.

Automate everything—People and processes are
error-prone. Automation will help avoid human errors. For example, “Minimize
Human Intervention” pattern discusses this in the context of telecommunication
networks.

Complex environmental factors—Distributed
systems must collaborate with services from different domains, handle
heterogeneous devices, complex configurations, and a large number of users.
Consider factors around latency, network failures, and service-level
agreements.

Analysts and architects should consider business and operational
behaviors and model individual services, business processes, and service
aggregates. These behaviors are documented as part of service models and
communicated to developers to facilitate instrumentation. Standards are
emerging to codify these requirements and constraints including Common
Information Models (CIM), Service Modeling Language (SML), and Web Service
Distributed Management (WSDM).

Design for Operations Patterns. Instrumenting services as
operations-friendly is complex. Developers should instrument services and
business processes with health events, measurement points, performance metrics,
provisioning interfaces, and operational support metrics. These patterns are
emerging to provide better guidance on how to instrument systems for
manageability.

Operational-Management Patterns. Operational systems need
management models, key performance indicators (KPI), and associated events to
measure operational intelligence. These systems resemble the observer-event
patterns or control bus pattern to define the interactions of managed entities
(such as Web service) and management servers (such as System Center Operations
Manager). For example, Management Packs will help codify the system
requirements, correlation methods, and automation steps. SCOM will push or pull
data from Web services to understand service health. Operational systems should
monitor continuous availability, performance, and degrading conditions of
service. Systems should provide capabilities to aggregate log files, send
health-check events, and service failover methods. Standards such as
WS-Management provide a way for systems to access and exchange management
information.

As Figure 5 shows, an architect defines management models, based
on the system’s operational requirements; developers add code instrumentation;
and systems administers look for events and patterns to take necessary actions.

Figure 5. Role-based management instrumentation

Table 3 lists the patterns categories in the service-management
domain.

Table 3. Service-Management Patterns categories

Patterns category

Problem

Solution

Operations-friendly service-modeling pattern

How to collect nonfunctional and operational requirements/constraints
and document these requirements

Use modeling tools such as Visual Studio
Architect Modeler.

Use System Center Management Model Designer
Power Tools.

Design for operations patterns

How to instrument in code for service manageability

Develop services using well-enabled service
principles using WES Toolkit.

Develop instrumentation using Design for
Operations tools.

Operational-management patterns

How to enable services or composites to be well-behaved

How to monitor business activities

Use SCOM for management.

Use SCOM Console to author/configure
management packs.

MSE to collect system messages and events.

Enable BizTalk BAM for activity monitoring.

Conclusion

In this article, we discussed a number of patterns that have been
observed during customer engagements focused on the development of distributed
systems. Given our customers increased dependence on technology and the
ever-increasing rate at which technology is evolving, it is in our opinion that
it is more and more critical to ground how we design and describe our products
in patterns. Without doing this, we increase the perceived complexity of moving
from a legacy technology to a newer technology. The patterns described here
were implemented in offerings such as DCS, ESBG, DCS, and the WSSF—all of which
are built on top of WCF. When Oslo is released, these same patterns will
continue to be valid with substantially lower cost.

We must take responsibility for capturing and sharing recurrent
solutions to problems such that our solutions and products can provide better
out-of-the-box support for core distributed-computing patterns. If you are
interested in learning more about how to do this, please contact the authors,
who are in the process of developing such a library of patterns.

The authors would like to
thank Blair Shaw and William Oellermann for reviewing an early draft of this
article.