Chris Waves the Red Flag

"There is simply no way that multi-threaded code can remain the defacto method of constructing concurrent applications."

The Problem

The tricks which deliver efficiency in the multithreaded emulation of parallelism made familiar by sound application programming practices on popular platforms do not efficiently leverage multiple cores.

Chris Waves the Red Flag

"There is simply no way that multi-threaded code can remain the defacto method of constructing concurrent applications."

The Problem

The tricks which deliver efficiency in the multithreaded emulation of parallelism made familiar by sound application programming practices on popular platforms do not efficiently leverage multiple cores.

The Idea

The idea is to turn your enterprise application into a workflow system which exhibits vector parallelism. Vector parallelism is probably the best model you can aspire to without advanced coding techniques. We're trying here to avoid advanced coding techniques and keep it simple and mechanistic.

If:

All work in the system can be factored in such a way that processing of a real-world work unit becomes internally:

A -> B -> C

And A - B - C are of roughly the same load

And A - B - C are parallelizable processes eligible under your chosen operating system and execution platform to be scheduled for separate processors and can thus run at the same time

Then you have a vector parallelized system that will scale nicely to the kind of multiprocessor server boxes and concurrent operating systems that are easily obtainable these days.

The Pattern

The Pattern (ooh, do I get to pick a fancy name?) let's call PQR for Process - Queue - Repertory.

That's the mechanistic beauty of it: it's three snap-together parts. Implement them as you wish, as suits you, as you are familiar with whatever tools you use.

Process

Does preferably One Thing and One Thing Only

Factor into evenly sized work units and the operating system scheduler is entirely adequate for realtime.

Simplicity is a virtue in coding as in life.

Gets its work from an In Queue

Which is the Out Queue of another Process or a Queue used for Dispatching to Processes

Outputs to its Out Queue

Which is, of course, the In Queue of some other Process or the return Queue to a Dispatcher

Queue

Connects Processes

Three kinds

In

Out

Work output of a Process

Is effectively the In Queue of another Process

In larger systems the PQR pattern can be extended to have Services (PQRS, get it!?) which are channels that dispatch to groups of Processes.

Command

System commands to individual Processes

Repertory

Local persistent and transient data that belongs

to an individual Process

to a group of Processes

Implement this as an in-memory cache that behaves like a

Database

Tuple store

The Benefits

Again, PQR is mechanistic. Were not looking for fabulous theoretical advantages; instead, we're looking for something:

easy to code

that runs fast

easy to understand

easy to maintain

distributable across multiple platforms

message architecture

locality of Repertory

offers small scope to typical parallelization errors

Good factoring itself assures that under a decent operating system scheduler the Process loads will be reasonably well balanced. Using Queues as the synchronization method assures correctness: if you can draw a map of your Processes and Queues, you can prove correctness of a PQR design.

The Implementation

Here's how you do PQR:

Grab some queuing middleware

Grab some in-memory cache middleware

Write in a compiler language without its own scheduler

Just leverage the operating system support for multiproc scheduling

If the operating system isn't sufficient, change operating systems

You see? You parallelize without ever having to write another lock, another spinner, another semaphore. All the nasty, error-prone parallelization code is in the message queue middleware. You've just built a giant chutes-and-ladders system where the little balls (work units) roll down the path and get dispatched to other chutes down other paths, with a small and carefully factored bit of work getting done at each landing.

The Case Study

In 2006-2007, Absolute Performance, Inc., (a profitable, privately held vendor of enterprise monitoring software targeted principally at SAAS operations ) accepted my recommendation to apply the PQR pattern to Version 5 of their "System Shepherd" ™ backend server. This represented a complete recoding from the Version 4 multitasking system which had been polished over a period of 7 years.

The software development team was expanded to eight (8) coders. The pattern was explained and design tasks shared across the team, with those having the most design experience focussing on workflow and those with junior experience tasked with implementation details and middleware qualification.

The project required that components of the Version 4 server be incrementally replaced by the new PQR-style Version 5 code. Partly because of the amazing skills of the motivated team and partly because of PQR's emphasis on factoring (PQR is all factoring, that's all it is) this design goal was emphatically achieved. Customers were serviced with the transformational code which, while occasionally lacking advanced features which were not yet supported, nonetheless executed reliably with easily predictable performance characteristics.

Won't bore you with benchmarks, let's just say Absolute Performance's System Shepherd now-complete distributed Version 5 runs Amazingly Faster than Version 4, and thus is able to carry a vastly greater load with the equivalent hardware with distributability so far limited more by the cost of test lab hardware than by any intrinsic limit to distribution in PQR itself.

A Few Hints

It still helps to read up on parallel systems.

Dust off those course books from your comp sci classes!

Your PQR project will be a successive modelling project

That means you will build increasingly more load capable increments of the planned system in incremental installments

Careful measurement is required!

You must profile your system and determine where the load is as a verification of the correctness of your factoring exercise.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!