November 26, 2008

On Complexity & Emergent Behavior

There is a fundamental problem with building efficiencies into
systems. There are essentially, with regard to white collar
productivity, two schools of thought, Deming and Hammer. Deming for
constant quality improvement and Hammer for business process
re-engineering.

The Deming approach requires a certain understanding of the process
in which all workers are engaged. From the standpoint of a software
designer, it can all be conceptualized in terms of a workflow. Person A
has tasks 1,2,3 Person B with function X has tasks 4,5. Task 4 requires
2, etc on down the line. The willingness of software engineers to
absorb the details of workflows in business is highly limited - and the
ability of managers of such processes to communicate their complexity
and possible efficiencies and chokepoints, etc. is also highly limited.
Furthermore the willingness for software engineers and business
managers to deal with each other on a long-term basis is slight. Both
see their time more profitably spent elsewhere. This is especially
costly to the Deming scenario, and so what generally happens is a
Hammer implementation.

This does damage to the process by obviating some complexity. So you
actually have more complex and new systems shoved into a process which
is actually dumbed down because of the communications limitations of
managers and engineers. Then what ends up happening is that those
employees in the process chain make up the difference by partially
working with, and partially working around the new system in ways they
have no incentive to explain, and often in ways their managers are
unaware of. The outputs of these systems as well as their inputs may be
manipulated because those processes to which information technology is
applied are rarely end to end systems. So there is a complex adaptive
dynamism at work and in the end describing these systems are like
describing the flow of molecules of water down a stream. They are
chaotic, somewhat random, extraordinarily complex, yet at a macro level
comprehensible and predictable.

Once a system becomes predictable at a macro level it gets frozen
into place as the practice of the business with all of the
inefficiencies and mysteries locked in.

And then the boss changes and all of the internal stresses on the system lose particular incentives.

Systems engineers generally don’t have patience for such matters as
conventions of using three employees to do a job that one person with
an improved system could do. In the process of conceptualizing the
system they are responsible for improving through information
technology, they will be introduced to more information about the
process than those people we call ‘functional people’ aka staff. This
generally causes tension and a resistance for staffers to be
forthcoming about those skills which be rendered obsolete.

Furthermore systems engineers have incentives to build generally
applicable systems. Laziness is good quality, that is to say that
re-usable systems which might be applicable to multiple businesses and
their processes are always seen as more valuable to one-off custom
systems.

Business managers generally don’t have patience or tolerance for
such systems which make transparent the operations of their staff.
There are often common sense improvements that can be made which
exposes them or their superiors to an embarrassment.

In general, business process improvement / re-engineering projects
and deliverables benefit most from being transparent and
self-explanatory. You want God powers immediately and you want control
over as many aspects as possible. However once these aspects of the
system are established you want them to respond to the changing nature
of the business. There is a fundamental conflict between thoroughness,
time to implement and adaptability. Pick two.

Gaming environments on the other hand tend to be hermetic, even in
sandboxes. There are certain aspects of the system which gives it a
specific feel which must be maintained in order to keep it interesting
and within a genre. The value of a game is in having a play value that
keeps someone progressing towards a God level which must be
entertainingly hidden. Emergent behavior in games is entirely
desirable. In business systems it is discouraged.

Comments

There is a fundamental problem with building efficiencies into
systems. There are essentially, with regard to white collar
productivity, two schools of thought, Deming and Hammer. Deming for
constant quality improvement and Hammer for business process
re-engineering.

The Deming approach requires a certain understanding of the process
in which all workers are engaged. From the standpoint of a software
designer, it can all be conceptualized in terms of a workflow. Person A
has tasks 1,2,3 Person B with function X has tasks 4,5. Task 4 requires
2, etc on down the line. The willingness of software engineers to
absorb the details of workflows in business is highly limited - and the
ability of managers of such processes to communicate their complexity
and possible efficiencies and chokepoints, etc. is also highly limited.
Furthermore the willingness for software engineers and business
managers to deal with each other on a long-term basis is slight. Both
see their time more profitably spent elsewhere. This is especially
costly to the Deming scenario, and so what generally happens is a
Hammer implementation.

This does damage to the process by obviating some complexity. So you
actually have more complex and new systems shoved into a process which
is actually dumbed down because of the communications limitations of
managers and engineers. Then what ends up happening is that those
employees in the process chain make up the difference by partially
working with, and partially working around the new system in ways they
have no incentive to explain, and often in ways their managers are
unaware of. The outputs of these systems as well as their inputs may be
manipulated because those processes to which information technology is
applied are rarely end to end systems. So there is a complex adaptive
dynamism at work and in the end describing these systems are like
describing the flow of molecules of water down a stream. They are
chaotic, somewhat random, extraordinarily complex, yet at a macro level
comprehensible and predictable.

Once a system becomes predictable at a macro level it gets frozen
into place as the practice of the business with all of the
inefficiencies and mysteries locked in.

And then the boss changes and all of the internal stresses on the system lose particular incentives.