The Insurance Plan

Over thirty years of professional experience designing and developing high performance parallel transactional server side systems in "C", "C++" and Java on *nix and Windows platforms for military, financial and health platforms. Bachelor's in Geological Engineering and Masters in Computer Science.

When designing a new feature of an application, among other things, you always want to decide how it will be used. Is it single threaded or will it need to happen in parallel. Will only one user do it at a time, or does it need to support asynchronous access. Will every user want to do it in the same way, or will they each want something just a little different.

Charlie C. worked for a modestly sized financial startup that had gained some traction. The company had grown to about 100 people. They had garnered about 300 customers, and they were building software that would solve a problem that was causing regulators all manner of headaches.

The system essentially provided the ability for customers to swap bond insurance contracts. These swaps could be between any two, three or four parties, depending upon the type of transaction. One day, one of the customers asked the CEO if they could implement an approval workflow. Specifically, before letting some low level trader execute a swap, it would be routed to a group with approval authority. If approved, the transaction would be routed to the other party. If not, it would be rejected back to the initiator.

The CEO asked the development manager who came to Charlie and told him to just hard code some flags in the main transaction record, and check for them as special cases at every relevant place in the code. Charlie knew that if one customer wanted it, that most (if not all) of them would want it. Accordingly, he explained as much and said he would make it table driven.

"Won't that take longer to build?" the boss inquired.

"Of course it will take longer, but it's a whole lot more supportable than special-casing the code all over, and corrupting the main transaction record," Charlie replied. With 300 customers and transactions involving 2, 3 or 4 parties, there were almost 8 billion permutations; even a tiny fraction of customers requesting special handling would make it absolutely unsupportable. "Think of it like… insurance."

The manager insisted on the hard coding, but Charlie ignored him, knowing all too well the consequences of listening to his boss.

A few months later, the work was done. On the very last day of work, as Charlie was doing some final cleanup, his boss walked over with the CEO in tow. They were both in a bit of a panic. "Another customer has asked for special workflow processing, but in a slightly different way. How much time will it take to add the additional workflow logic?"

Charlie looked at his boss, laughed, and said "About thirty minutes."

"But it took months this time!"

"And this is why I don't listen to you." Charlie went on to explain that it would take about a half hour to create the relevant groups, users and flags to simulate the new workflow, and that he would try to start the servers, queue managers and 16 copies of their application (one for each user in the scenario) on his PC to demonstrate that it would perform the required routing.

At the appointed time, both CEO and development manager returned and watched as Charlie made his machine groan through the motions to perform the desired flow processing. While it was busy thrashing, he used the opportunity to point out that they had no demo environment, that the developer workstations were simply not powerful enough to perform these sorts of demos, and he apologized for taking so much of their time. As the CEO listened to the hard disk grinding away, he agreed and offered to let them buy more powerful developer workstations.

After about an hour of going through all the different scenarios, the CEO was pleased, thanked Charlie for his time and told him to put in an order for the workstations the next day. Then they walked away.

Later that day, the development manager was transferred to manage another team and someone else was put in charge of Charlie's project. Shortly thereafter, Charlie grabbed the other developers and put together specs for their dream machines, which were ordered the next day.

Over the next year, configurations for more than a thousand different special case workflows were added, and not a single byte of code had to change.

[Advertisement] Manage IT infrastructure as code across all environments with Puppet. Puppet Enterprise now offers more control and insight, with role-based access control, activity logging and all-new Puppet Apps. Start your free trial today!