SATURN Conference 2015

May 1, 2015

ABOUT ME

Hi, I am Gregor Hohpe, co-author of the book Enterprise Integration Patterns. I like to work on and write about asynchronous messaging systems, service-oriented architectures, and all sorts of
enterprise computing and architecture topics. I am also a technical director in Google
Cloud's Office of the CTO.

I have not blogged about events in a while (my Rambling category tells me almost 6 years!), but SATURN 2015 in Baltimore has been an amazing event that's well worth rambling about.

The perfect blend

One interesting aspect of SATURN is that academics call it an industry conference whereas folks from industry tend
to call it an academic conference. While this can lead to minor annoyances (yes, it
is an industry conference), I feel it highlights why the event is so unique and useful:
it is a perfect blend of structured thinking from the academic edge combined with
valuable industry experience that would otherwise not make it to an academic conference
due to lack of analysis and validation. The blend worked extremely well for me. It
also implies that there is no product or vendor pitching at this event, aside from
the occasional book pitch by the author (which should be acceptable - you would do
the same after going through all the effort) and brief mentions of the sponsors, including
Allianz.

Microservices Workshop

Monday was an all-day microservices workshop setup by George Fairbanks and Michael Keeling. One may feel that there is not much
left to say about microservices with Sam's Book out and Martin bliki-ing about it. Still, thanks to the superb moderation and the caliber of attendants I
found the workshop to be very useful. We split into subgroups, of which I attended
the one on "API Design", always a topic of interest. We discussed way too many things
to even fit into a rambling, so here just some sound bites:

Interaction Models: API design focusses too much on data representation and too little on interaction
models. I am trying to help solve this.

Domain-driven Service Design: For the data representation aspect and the "slicing" of micro services we should
have a fresh look at Eric Evans' Domain-driven Design. Much of his guidance should apply to service design as well. For example, domain
boundaries might make good service boundaries.

Versioning: Service API's should be immutable, relegating evolution to new versions. Another
option is to do what Google did with Stubby and ProtoBufs's: they push the versioning
problem into the application layer by extending existing data formats with optional
fields and designing services such that they can accept calls from "old" callers who
do not yet supply the new fields. In the easiest case this is done with defaults,
but it can require the service to check which fields are present and to execute the
corresponding version of the service. It's clever, but from my experience it depends
on thorough documentation of the behavior inside the proto definition files.

It's all connected: While this sounds like the dream of a person enamored with integration, at the logical
level things are indeed also interconnected: the current complex systems and rapid
cycles have too many moving parts and too many client versions to rely on complete
testing upfront. Therefore you can only test in production, e.g. with canaries (deploying
new versions to a limited number of servers to see how things go). This raises the
question whether users are happy if they experience different versions of a service
or web site, sometimes even ahead of announcement. If your changes are small and incremental,
it works relatively well. You can have small changes only if you have short release
cycles -- otherwise you are making no progress. This implies that out of the Micro-services
/ Agile / DevOps mix you cannot arbitrarily pick a subset - its "all in or nothing".
Martin makes a similar argument in his You must be this tall to use microservices article.

Governance: Do you let 1000 flowers bloom or do you exercise central control? To me this sounds
like a classic decision analysis question: If you decide upfront and exercise governance you have lower cost -- as
long as you guess right. If you guess wrong you face high rework costs. Letting things
evolve over time has a higher inherent cost, but also a higher chance of leading to
the "right" answer. Governance also takes many more forms than a fat book and a big hammer. For example
Netflix (all good exampels these days seem to come from Netflix) provides a client
library that include circuit-breaker functionality. Of course not everyone uses that
library. So they run little disturbances in production that causes your service to
get hung -- this helps adoption!

Dynamic properties: Mary Shaw reasoned that systems using micro services architectures are emergent,
meaning you cannot reason about their behavior until they are assembled and running.
She would place them in the "implicit invocation" block of architectural styles because
the invocation of one method by another is not explicit like in OO systems, but implicit
like in pub-sub systems.

Monitoring: In such a dynamic system, monitoring is paramount. Len mentioned that Netflix collects
90 metrics for their canaries.

Mary Shaw Keynote

In case you did not know, Mary Shaw co-authored the first and seminal book on Software Architecture with Davie Garlan. A key aspect of the book is the categorization into "architectural
styles". For example, we cite the Pipes-and-filters style in EIP. She described that the concept emerged when she collected a large number
of technical documents which tended to include roughly 6 column-inches on "the architecture
of my system" supplemented by about 2 column-inches of diagrams. Looking at a large
number of these sections, certain categories emerged, which after a while solidified
into architectural styles. Harvesting design patterns is actually a quite similar
process of looking for common approaches, although the nuances are somewhat different:
patterns dig deeper into the forces and intents of the recurring solutions.

Mary (unsurprisingly) talked about software engineering, with the core thesis whether
software engineering does in fact deserve to be called engineering. She drew on civil
engineering, specifically bridges (Cindy would be happy). It's actually quite amazing
that the methods devised by the Roman's without any theory and very little mathematics
lasted into the 19th century. A key element to their success was empirical progress:
they learned from failure and kept refining their concepts. So real-life buildings
can be agile, at least if you have enough time and cheap labor. If you want to add
a true classic to your bookshelf, get Vitruvius' De Architectura from 25 BC. The science of bridge engineering came much later, e.g in the form of
tabulation of iron properties in the mid 1750's. A hundred years later engineers were
able to do a full analysis of a bridge, at least minus resonance.

So how does software engineering compare? In oder to start from the right mental model
we have to remember that software is design intensive with negligible manufacturing
cost: coding is design. Manufacturing used to consist of printing CD's but now consists
mostly of uploading binaries to a distribution site. People often feel that there
is no science software development or software engineering. Mary showed that there
is quite some science in software; it just takes 15-20 years from basic research to
popularization, at which point it is no longer called "science". In our world much
basic research happened in the mid-80ies, e.g. the notion that connectors are as important
as components. I guess it's time I finish my interaction patterns... The early 90ies
saw concept formation followed by much development, exemplified in many conference
tracks on software architecture and engineering. Internal exploration brought focus
on formal analysis, quality attributes, and methods such as ATAM. External exploration brought us processes such as RUP and component-based software
engineering. Popularization was reached in the last 15 years with many standard interfaces
and frameworks, many of them driven by the evolution of the Web.

But software engineering still falls short in many aspects. "I am trying as hard as
I can" is not engineering, Mary reminded us. We fall extremely short in making reference
materials available. Every chemical engineer can reach for his copy of Perry's Handbook
-- we have no such thing. Even Grady's effort on the grand Handbook of Architecture
seem to have stalled. One key difference is that Perry's Handbook has a 15 year update
cycle, something that would not do us much good (side note: it looks like EIP is heading
for a similar update cycle). Search has become our method of sharing knowledge. While
this is a fact of life we should consider whether crawling discussion groups is really
the path to more mature and reliable software engineering.

Sam Newman on Microservices

It was great to see Sam again and reminisce about old stories including pasta tongues
imported from Japan. He gace a great overview of what it means to use microservices.
Rather than rehash it here I recommend you read his new book.

Later Simon and Len put Sam on the "Microservices trial", which yielded a ton of great
sound bites and pearls of wisdom, including:

Box-box-cylinder as architectural style (Simon)

Monolith vs. microservices style is like America vs. Europe (Les)

Using the network enforces discipline in setting boundaries (Sam)

How would you use a chaos monkey with a monolith? You'd have to take the whole system down! (Sam)

We should be cautious with standards. UDDI was a "lets solve a problem we have not
solved yet" type of standard (Sam)

If you have a brand-new product, first build a monolith, because you won't know where
the seams are. Then go to microservices later. (Sam)

Software Architecture as Code

Simon Brown and I bonded on software visualization, especially aided by annotations.
I should visit Jersey and he should talk with Erik Doernenburg.

80 Years of Patterns Impact

SATURN was also a great moment to bring together the 4 authors of our 20 years of patterns' impact paper: Rebecca, Joe, Olaf and myself. I am still entertained by the thought what the odds
are that you can author a paper with 3 co-authors whose names start with W (Wirfs-Brock),
Y (Yoder), and Z (Zimmermann). Bobby should talk to some of these guys :-) The 80
years are maybe a slight exaggeration as for example I was not part of PLoP until
2002 when we workshopped EIP. The picture shows us in correct alphabetical order.

Thanks to all for a great event. There were many more great talks and conversations
(and some Belgian style brews) that I did not manage to write up. And we stayed safe
despite the ongoing protests in Baltimore. In the end we only saw a single and very
peaceful protester on the way home.

Last, but certainly not least, I should thank my muse for inviting me to SATURN!

Gregor is a technical director in Google Cloud's Office of the CTO. He is a frequent
speaker on asynchronous messaging and service-oriented architectures and co-authored
Enterprise Integration Patterns (Addison-Wesley). His mission is to make integration and distributed system development
easier by harvesting common patterns and best practices across many different technologies.www.eaipatterns.com