Saturday, May 7, 2016

In my earlier post, A Financially Frugal Architectural Pattern for the Cloud, I advocated the use of microservices. Microservices are similar to REST, a concept or pattern or architectural standard, unlike SOAP which is standards based. The modern IT industry trend towards “good enough”, “lip-service” and “we’ll fix it in the next release”. A contemporary application may use relational database software (SQL Server, Oracle, MySql) and thus the developers (and their management) would assert that their is a relational database system. If I move a magnetic tape based system into tables (one table for each type of tape) using relational database software – would that make it a relational database system? My opinion is no – never!!!

Then what makes it one? The data has been fully normalized in the logical model. Often the database has never been reviewed for normalization despite such information being ancient (see William Kent, A Simple Guide to Five Normal Forms in Relational Database Theory, 1982), older than many developers. The implementation may be de-normalized in the physical model (if you have just a ‘database model’ and not separate physical and logical, then you are likely heading to trouble – in time (usually after the original developers have left!). For NoSql database, there is a lot of ancient literature out there dealing with both hierarchical databases and network databases which should also be used with MongoDB and Neo4j – but likely not.

My academic training is in mathematics and thus axioms and deriving theorems from them though rigorous logic. The normalization of databases is immediately attractive to me. Knowing the literature (especially Christopher J.Date’s early writings from the 1970’s) is essential since “"Those who do not learn history are doomed to repeat it.”

A Dependent Microservice

Create Update Delete calls must always go to the independent microservice that owns it, no relaying should occur.

Calls are in parallel, never sequential

Note: Some types of queries may be inefficient, those should be directed at a reporting microservice (which independent microservices may publish to) or a composite service.

Control of Microservices

The best model that I have seen is one that some groups at Amazon used.

All calls to the microservice must have an identifier (cookie?) that identifies the caller.

The microservice determines if it is an authorized caller based on the identifier and possibly the IP address

The consumer of the microservice must be authorized by the owner of the microservice based on: a contract containing at least

Daily load estimates

Peak load estimates

Availability

The microservice may disable any consumer that exceeds the contracted load.

The consumer should be given a number from 1-100 indicating business importance.

If the microservice is stressed, then those services with lower values will be disabled.

There should always be SLA in place

Microservices should be executed on isolated VMs behind a load distributor. The datastore should be also on a dedicated set of VMs, for example a set of VMs supporting a Casandra implementation.

More suggestions?

To quote Martin Fowler, “If the components do not compose cleanly, then all you are doing is shifting complexity from inside a component to the connections between components. Not just does this just move complexity around, it moves it to a place that's less explicit and harder to control.” I have seen this happen – with the appearance of microservices (because there are tons of REST APis on different servers) but behind this layer, there are shared DLL’s accessing multiple databases causing endless pains with keeping DLL current as features are added or bugs fixed. A bug in a single library may require the fix to be propagated to a dozen REST APIs’. If it must be propagated it is not a microservice.

My goal with this post is to define a set of objective check items that can be clearly determined by inspection. The ideal implementation would have all of them passing.

One of the side-effects is that the rules can often be inconvenient for quick designs. A rethinking of what you are trying to do often results – similar to what I have seen happen when you push for full normalization in a logical model.