Introducing Yoctoservices

I know a guy, let’s call him Peter, who knows a guy. Let’s call this last guy
Jim. Their connection was mostly professional, as Peter was an engineer working
for the same company as Jim.

Jim, unlike yours truly, is not a software practitioner. Jim, in fact, is
“Principal software architect” and Jim’s first and foremost responsibility is,
as you might have already guessed, software architectural design and
development. Which involves almost incomprehensible amount of theoretical
knowledge, which Jim amassed in full by having filled his Drive almost
exclusively with endless PDF’s and E-Books on the topic. Jim’s collection might
or might not make some librarians jealous.

No, no, no, don’t you make that face just yet, software architecture is a good
thing, we are getting there. I thought Jenkins pipelines have already taught you
some patience, no?

Apart from tasks and duties of your usual God emperor of the known uni… I mean,
Principal software architect, after an impressive list of other
responsibilities, duties and literally everything came Jim’s duties as the
de-facto head of The Team, which compromised of Jim himself, project manager
Billy, and two developers, Sam and Tina.

Jim has spent couple of years working in American Midwest for “Your Usual
Consulting and Kickups Inc”, commonly abbreviated as YUCK. YUCK was by no means
a small company, having engaged in pretty much every industry it could reach
with it’s apocalyptic tentacles, ekhm, also know as sales and marketing
department.

Came the fateful Tuesday morning and deliverance by Billy, the PM.

— “Hey Jim, listen, you remember that lease tracking software we did for our
generic car rental customer last year?” Billy said prompting Jim to, as usual,
not pay any attention whatsoever. “They have expanded their business since and
they now have more shops, so they would like to have the particular piece of
software available online in all of their locations, interconnected as to see
availability of stock and such” Billy continued “so, what do you think? Not a
pressing issue but… “

— “Microservices”.

— “What?” Billy uttered, mostly in disbelief for actually getting some life
signs out of usually dormant Jim.

— “I said micro… services. We will build a microservices based web platform for
them. I remember the requirements, should take no time at all!” Jim explained,
and to Billy, Jim sounded almost happy. Excited even. Billy stared in disbelief,
as if stricken by most electric kind of lightning right then and there, and
started to contemplate about what has just happened. It usually took about
infinite amount of weeks to get Jim past planning and design stages, but this?
No design diagrams? No meta programming? Just “should take no time”?

— “Are you sure, Jim? This is not something we can build from scratch, existing
data needs to be preserved, plus we need to merge it from all of their local
installations” Billy inquired, unsure of what to expect next. Ragnarok?…

— “Yeah, I know, I know what they need, don’t worry. This is an entirely new
concept I have read about and I really want to try it out on a production
product. I already have the solution and they are going to absolutely love it. I
will have it by the end of the month” Jim waved “I really have to finish this,
do you mind…?”

— “No, of course, carry on then, I will email you the details” Billy shrugged
and started her way back to The Cubicle.

Days passed, weeks went on, Jim designed and worked on The Architecture, while
Sam and Tina all in all where mostly left wandering around countless source code
repositories, “lambda” SDK’s and other interesting concepts that seemed to flow
like never ending river, backed by almost exclusive and religious quarter-hourly
SCM commits by Jim.

— “So, what’s the latest?” Billy inquired to Jim.

— “We are almost there, just a couple more services. We had a couple of bumps
along the way, but we are about to be there” Jim retorted, business as usual.

— “How are we on storing their current data in this new platform?” Billy
inquired again, hoping for the same answer.

— “What do you mean?” Jim asked, face on as if Billy had just spit in his
morning coffee.

— “Well, their production data, remember? We have to load all the customer info,
all of it, I sent you the samples weeks ago”

— “This is not possible. It’s microservices, we can not. They will have to type
it by hand”

— “Can we just write some script to transform the structure and import it in the
new database?” Billy raised an eyebrow

— “No”

— “What do you mean no, we need to, I mean, customer needs it and…”

— “Billy, we can explain to the customer this is a new kind of technology, we
can not, it just does not work that way. They will need to type it in by hand”

— “This will take months, this is not what we agreed upon, it needs to happen,
and that’s that”, Billy’s frustration grew and patience decreased with each
sentence Jim produced.

— “No”

— “Tina, Sam?” Billy addressed the rest of the team in hope of salvage.

— “It’s just technically… Not possible. Sorry Bill’. We…”

— “So what you are saying is, I need to call the customer, inform them about how
we spent all this time developing this… thing… and now they need to spend as
much on data entry?”

— “Yes”

— “No”. Exit stage Billy.

Days went by, and as you, my dear reader, might imagine neither Billy, nor the
customer representative was very happy about the result. Names where called,
blame assigned, directors of both sides involved, monetary losses calculated.
This is how it would end, unless the mastermind of Peter’s manager to allow
their team to try and unravel the mysterious microservices platform Jim had
spent all this time architecture-ing away. This task fell on Peter’s shoulders
as the most senior developer of the team.

Disassembling the future.

— “What the f**k?”

— “Why would you?”

— “What?!”

This is Peter. Peter is a software developer. Peter, as you might imagine, is
currently a little bit frustrated. Just a tiny bit.

— “Hey Tom, come here. Look at this”

— “What’s up?”

— “I can not believe this piece of s**t, I mean, I have seen things, but this.
Just wait, look, tell me Tom, what do you see here?”

— “A repository listing, why?” Tom shrugged.

— “How many pages of repository listings do you see, Tom?” Peter inquired half
enraged, and half laughing about the absurdity of the situation. But mostly
enraged.

— “17. Why? Wait, what project is this?”

— “10 repos per page, 17 pages, all in all, about 170 repositories.” Peter
replied, as if struck a gold mine. “That’s the car rental project. One of
Jim’s”.

— “Yes, look at this s**t” Peter expanded couple of the repositories “Each one
of these contains some kind of service if you dare to call it that way, each
repo contains an app that basically runs an embedded version of web server or
whatever that platform provides. “ Peter continued his rampage hitchhikers guide
to what could only described as mayhem “These apps seems to talk one with
another in some kind of obscure HTTP based RPC protocol, but not always, there
seems to be a couple of TCP servers running, for a purpose I can only guess.

— “Then there is the databases. Each of these apps has their own database, and I
don’t mean different logical databases, it’s different instances AND different
database engines altogether. Like this Express app seems to interact mostly with
Mongo, while this one,“ Peter pointed his finger at the source repo like a
naughty child “this one here uses Postgres. And this one uses Elastic for some
reason.”

— “Now look, I had to local install this to actually see IF and how it even
works together, and of course, I had to install all of the dependencies, most of
which where undocumented by hand, because why the f**k shouldn’t I”

— “Now this is a great example on how this works, the customer rep system. It
consists of some 30 of these apps bound together, and the actual web is built by
some 5 of them as far as I can tell. The login system is especially interesting
design. You enter your credentials, those credentials get sent to another one of
these, the *login_auth *app, here, and that app creates something called
“AuthenticationTicketRequest”.”

— “That get’s you an auto generated ID in response, which then get’s sent back
to the login app, where it is passed to UI and then the user get’s redirected to
another app, called login_verify, this ticket request thing ID in the URL.
That service then compares your credentials with another service, called
user_store, this is in back end, via RPC request. “

— ”If your credentials are confirmed, you get a session ID which then get’s
attached to the URL as you get redirected back to the app you were initially
going to. Then, whenever you ask for any resource on the server, and I mean any,
including what appears to be most static assets, which, by the way, have
services for retrieval and compilation, and what not, of their own, you must
send this token along with the request via GET or one of many headers defined
for this purpose.”

— “Then, for each request, whichever app you access to, it invokes RPC function
in the background to the user_store to validate the token. If the token is
identified as invalid, you get kicked back to the login page.”

— “On top of that, there is a roles management system as well. All would be nice
and well, but there is a separate service for that. Whenever you need to access
some section or whatever kind of isolated feature, you, well you guessed, you
need to make a RPC call to the remote service to validate access. Each section
makes tens if not hundreds of these, every single time you access it. No caches,
in fact, it was explicitly documented to not cache anything because… someone
could change the access rules”.

Peter took a pause to evaluate expression on Tom’s face after this epic
monologue. As you expect, Tom’s facial expression was an equal mix of “Wut”,
disbelief, disgust and serious reconsideration about his career choices, all in
one.

— “He actually described and named this whole thing in the application docs
somewhere. Yoctoservices. Apparently it is advanced kind of microservices.”

Bottom line

On a bit more serious note, stop. Stop *microservices *literally everything.
Microservice architecture is good for some very specific applications, as is
pretty much everything SOA. No, your corporate with average reader base of 5 per
eon does not necessarily need microservices to sustain it.

If you are not able to explain why you should use microservices as base for your
arch without using “scalability”, “decomposition”, “isolation”, “performance” at
all, you do not need microservices.

If you can not explain why you need microservices in less than one paragraph,
you do not need microservices.

Chances are excelled your microservices will be even bigger pile of crap than
your aging monolith, mostly because you underestimated problems a networked
infrastructure might and will have, and you greatly overestimated your own
ability to design, develop and maintain large scale distributed systems. It’s
like drunk driving. Everything is unicorns till you hit a tree doing 120mph.

Disclaimer

Let me just end this rather nicely assembled article by saying, I am a software
practitioner. I always have been, and always will be.

I have, in my entire career, read exactly zero books about software design. I
have, in my entire career, written exactly zero books about software design. I
have no intention to do so either.

In fact, my personal library consists almost exclusively of project management,
operations and strategic management textbooks, and related literature, most of
which is sourced from NYU/Stern, MIT and anywhere in between.

When I say I am a software practitioner, I want you to comprehend fully what
exactly do I mean with that, and that is, I give less than negative infinity of,
you know the f-word, about what your typical highly paid consultant slash
speaker slash author thinks about how I, or anyone else for that matter, should
write software. There are very limited number of exceptions to this rule, but,
as usual, you should assume the worst. And no Martin, you are not one of them
‘xceptions.

On the other hand, I have met, and even better, had the opportunity to work
alongside with some incredible people, who like me, are practical people and
does not seem to care much about theoretical issues, people who know what they
are doing, how, why, when and when not, and why when not. If you ever read this,
you know who you are. I thank you. I have learned from you so, so much more than
I could have ever imagined.

Also, as you might have noticed (at least I hope so), I care rather very little
about political correctness. If any of the contents offended you in any way,
deal with it.