Reflecting on Enterprise Integration Patterns

January 1, 2008

ABOUT ME

Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I like to work on and write about asynchronous messaging systems, service-oriented architectures, and all sorts of
enterprise computing and architecture topics. I am also the Chief Architect at Allianz SE, one of the largest insurance companies in the world.

The end of the year is always the time to reflect on the past happenings. My friends
in Japan often send a New Year's card with 12 pictures, each showing the significant
event during the month. I am not sure something significant related to Enterprise
Integration Patterns happened each month, but I think it's still nice to reflect a
little bit on the history and current state of the patterns. After all, the patterns
have matured from a draft paper to being adopted as the lingua franca for a number
of open source messaging projects.

Pre-Dawn

I have been working in integration since the ancient days, when "EAI" was still a
cool term. Even my very first client project was an integration project! I had also used Design Patterns extensively when building
large applications (I remember it took me a few attempts to understand the Visitor pattern).

Working with EAI tools of the day, it did not take long for me to figure out that
integration is difficult. Besides building solutions with the tools, I started to focus on best practices
to help clients better understand the development process and architectural trade-offs.
When I started working with multiple integration technologies (my company figured
that someone who knows Vitria should also understand TIBCO), it occurred to me that
deep beneath the surface (and a pile marketing material) these tools do share similarities.
There are message queues, transformations, process editors and engines, message metadata,
routing, and so on. But it had not dawned on me to document some of these insights
as patterns.

EIP Shotgun Wedding

The meeting between integration and patterns happened when Martin Fowler was putting
the finishing touches on PofEAA. Kyle Brown from IBM pointed out that enterprise applications often rely on messaging
(and IBM happens to sell a J2EE container including JMS), and suggested Martin should
include messaging patterns in his book. Martin, knowing a thing or two about publishing
books, quickly sensed the dangers of scope creep and decided to stick to patterns
inside the application (with a few cliff hangers like Gateway.

With everyone agreeing on messaging being a relevant, but under served topic, in early
2002 Kyle started a series of meetings in RTP to talk about messaging patterns. I remember that on one of our first meetings, North
Carolina was blessed with very rare snow fall, pretty much crippling all forms of
transportation. RTP is also the place of the "shotgun wedding" (Martin's choice of
words) between Bobby and myself to take on the task of documenting patterns related
to asynchronous messaging and enterprise integration.

EnterpriseIntegrationPatterns.com is Born

After we looked at the volume of work contained in the submitted papers, we decided
(with a lot of nudging from Martin) that the patterns deserve a more permanent home
on the Internet. Hence, the domain with the long name www.enterpriseintegrationpatterns.com was born. Because I am a poor typist (and the name is too long), I also registered
www.eaipatterns.com, and Kyle got www.messagingpatterns.com (we actually "lost" that domain to squatters for some time, but it is back in good
hands now). All domains point to the same content. We now receive a steady stream
of about a thousand visitors a day.

Lots of Editing

The next few months were characterized by a lot of feedback, revisions, and editing.
Our revision history captures some of this activity. A lot of discussion also occurred on our Yahoo! mailing list. The list has been overrun by spammers, but we'll try to clean it up so we can preserve
this historical record.
After the "open" review, we submitted the manuscript to select reviewers, which included
Luke Hohmann, David Rice, Richard Helm, and the Silicon Valley Patterns Group. Based
on the detailed feedback (and to our shock), the manuscript grew from some 400 pages
to 700.

Incorporating the technical feedback was followed by tedious months of proof reading,
image editing, and copy editing. That's when Martin Fowler's voice appeared to us
like Obiwan reminding us that every time you think you are done with the book, there's
more work left than you did up to that point. Even the force can't help there.

The Book!

Our lives returned somewhat to normal in summer of 2003 when the final manuscript
when to the printer. Finally, at OOPSLA 2003 in Anaheim, Enterprise Integration Patterns
hit the shelves. At the show, we were outsold by Eric Evans' Domain Driven Design,
but we were still mighty proud to see people actually buy the book. We even got to
autograph a good handful, much to Martin's astonishment.

Of course, the book has come a long way since then. We have well over 35,000 copies
in print by now, not counting the international versions. We still look at our Amazon Sales Rank every once in a while. We are also most pleased to report that 25 out of 29 reviews
give us 5 stars. Since feedback is important to us, I even created an RSS feed for our book reviews using Yahoo! Pipes. Interestingly, EIP also put on some weight over the years. No, we did not add anything
(I am busy posting ramblings and Bobby runs a blog at Developerworks), but it's simply the magic of paper weight.

Too Much to Say

Naturally, as soon as a book goes to the printer, the authors can't think of anything
else but all the things they would have liked to include in the book. There is always
more to say than fits in a single book. Thus, in October of 2003, these Ramblings started. Instead of having a traditional blog, my intention was to have start a collection
of small "articles", which cover relevant topics in the integration space. I was envisioning
something like my own monthly column. It turns out that I have not even managed to
post once a month on average, but I think I managed to cover some interesting topics.
The ramblings is also where we first learned about the relationship between making coffee and transactions, the rambling that still gets the most hits and was featured in Joel Spolsky's Best Software Writing.

The World Wild Web

Unlike most blogs, mine does not allow readers to post of comments. It's not really
by intention, it's mainly because my blog system is hand-rolled in XSLT (argh!) and
it was too much work to implement a comment-feature with spam detection and such.
A full-text RSS feed is also still missing. At least I made my feeds recognizable
by FireFox.

Speaking of spam, it's the thing that killed the Enterprise Integration Pattern Wiki.
As we often point out, patterns are not invented, but harvested from practice. Therefore,
it seemed great to have a Wiki with a page for each pattern that allows readers to
add their own experience using the pattern, or even add new patterns. This was quite
successful for a while – it was great to see people's input on our patterns. For example,
some readers added implementation tips or words of caution when applying certain patterns.
Sadly, spam wiped out all topics on the Wiki because I used a standard wiki with no
spam protection. I think I still have a backup. I hope to be able to find time to
reinstate at least some of the content.

Family

One of the greatest things about writing the book is the group of truly great people
I met through the work. All of these people are extremely bright, great communicators,
and simply fun to hang out with. The list of people I met is simply too long to be
fair to everyone, but I want to rattle a few folks off. Even in the early stages I
got to meet the folks from the Silicon Valley Patterns Group. You'll see their names in the "Acknowledgment" section of many, many good books.
Then of course, I got in close touch with many other popular pattern authors, including
Ralph Johnson, Martin Fowler, the John Crupi, Danny Malks, Deepak Alur triumvirate,
the Siemens Gang (Frank Buschmann, Michael Kircher, and many more), and words-do-not-suffice
folks like Kevlin Henney. The Hillside folks are a group of really neat people, including
Linda Rising, Joe Yoder, and Brian Foote (the last two of Big Ball of Mud fame). Of course, patterns are also a popular teaching tool, so I like to connect
with folks like Uwe Zdun and Schahram Dustdar from the TU Vienna, Wil van der Aalst
in Eindhoven, folks from the Enterprise Architecture Center at the TU Berlin, Gustavo Alonso at the ETH Zurich, and of course Frank Leymann in Stuttgart. Frank is a convert from another great group of people, the often underappreciated
folks who work on standards. Even I have dished out the occasional joke at the standards
bodies, but they include a long list of very bright folks who truly try to make solution
development less painful. I certainly can't list everyone, but Frank Leymann (WSFL
-> BPEL), Satish Thatte (XLANG -> BPEL), Sanjiva who-can-tell-me-how-many-vowels-his-last-name-has-wins-a-price (WSDL and many more), John Evdemon (BPEL), Jean Jacques Dubrai (ebXML, WS-CDL) spring immediately to mind. Open source is a major driver of innovation
these days, so I am very happy to have met many of the great minds behind the most
significant software movement of the last decade. Again, many omissions, but of course
the LogicBlaze / Iona folks (James, Bruce), the Mule team (Ross, Dan), and the Interface 21 gang (Rod, Rob, Alef, Arjen, Adrian, and many more) have to be near the top of the list
(it's particularly fun, because on one hand they are fierce competitors, but they
are also all great guys). Then there are the illustrious personalities like Hani and
Crazybob, plus the Fowlbots, including Paul Hammant, Alsak, Erik, Jon Tirsen, and many more. Oh, I am tired of
typing (I can’t touch type), so please fill in all the missing folks. I am also too
lazy to create links to everyone, but I can recommend a decent search engine to do
the job for you ;-)

Integration Patterns in Open Source

I often find myself explaining that patterns are more than a solution (the canonical
example are GOF's State and Strategy patterns, which have exactly the same solution structure, but very different intents).
Still, it is quite normal for frequently used patterns to make their way into common
libraries and platforms. Naturally, a number of our patterns were already implemented
in messaging middleware before we even documented them. Still, I am very happy that
a number of (commercial) open source projects have adopted our patterns. The most
relevant ones (in no particular order are):

If you know of other open source frameworks that use our patterns, let me know.

Universal Language

Enterprise Integration Patterns is also no longer restricted to English. We already
have a Russian and Chinese copy out, with a Korean one in the works, and a Japanese
version under negotiation. If all goes according to plan, EIP will soon be available
in five languages for you to enjoy.

What's Next?

People sometimes ask whether we'll do a second edition since many of the code examples
are now also four years old. My prediction is that the chances for a second edition
are slim. Almost all code examples use languages that are still in very popular use
(mainly Java and C#) and should compile with the latest versions of the respective
languages and frameworks. Yes, we are missing out on some neat language features (annotations
/ attributes, generics, etc), but for our small code examples those would not really
add a whole lot to the pattern. We would be happy to make all code examples available
in open source, which also allows readers to help maintain or update them. If I ever
have a spare minute, I'll set that up.

Of course, there are many more integration patterns to be harvested and documented.
Sadly, my progress on this front is extremely slow (it turns out you can't use 20% time to work on personal book projects). Last year, I held a workshop in Dagstuhl on conversation patterns, and just last month I workshopped a like titled paper at
OOPSLA, which documents over a dozen new patterns. With a little luck, we'll be seeing
some of the new patterns on this site soon.

Gregor is the Chief IT Architect of Allianz SE. He is a frequent speaker on asynchronous
messaging and service-oriented architectures and co-authored Enterprise Integration Patterns (Addison-Wesley). His mission is to make integration and distributed system development
easier by harvesting common patterns and best practices from many different technologies.www.eaipatterns.com