11. Neuronal Resonance and User Interface Technology

11.1. Abstract

Before the advent of power driven machinery, all human work
was governed by rhythm. The high art of tool and weapons design in the
pre-industrial age consisted of close-fitting the
weight-hardness-elasticity-impact characteristics of any tool to the
corresponding biorhythmics of the moving human body. In the terminology of
current neurological research, this art of tool design was aimed at perfecting
the Neuronal Resonance (abbrev. NR) feedback loops between the tool and the
human body. With the advent of power machines, the principles of NR design were
largely forgotten, with notable remnants in the field of sports gear, like
skiing, tennis, or golf equipment. In the field of computer technology, NR
design represents a veritable ergonomic "black hole", due to a compounding of
social, economic, and technological factors, and the presently dominating GUI
(WIMP) UIT can be justly considered as ultimate achievement approaching the
theoretical maximum of anti-NR design by single-channelling the human-computer
interaction bottleneck through a one-hand / one-finger pointing device: the
mouse. Only now are these insidious limitations and drawbacks of the GUI coming
to wider attention, as is evidenced by the great interest in the works of
Hiroshi Ishii, MIT.

The paper explores the philosophical, social, economic, and
technological backgrounds of the present NR deadlock, tracing it back to a
general oblivion of dynamic factors in ancient Greek philosophy (Parmenidean and
Platonic views dominating over Heraklitean), a general pre-dominance of visual
factors in Western European thinking, and the Platonic, time-neglecting attitude
of the foundational mathematical base of computer science, and its paradigm, the
Turing Machine as the ultimate serial device. Further, the "Lemming" effects of
a von Neumann player coalition between a monopolizing SW industry and the
"lowest common denominator" largest-possible user base, are explored. The
effects of cognitive enhancement through NR optimized design are a seriously
under-researched field, with the "flow effect" by Csikszentmihalyi being a rare
exception from the general pattern.

11.3. Table of Contents

11.2. Abbreviations

aka also known as

EUPL End User Programming Language

GUI Graphical User Interface

NR Neuronal Resonance

NRT Neuronal Resonance Technology

RSI Repeated Stress / Strain Injury

SW Software

UIL User Interface Language

UIT User Interface Technology

WIMP Windows - Icon - Mouse - Pointing UIT - GUI

11.4. Neuronal Resonance Technology: History and Development

11.4.1. The Pre-History of Neuronal
Resonance Technology

Human intelligence is the evolutionary product of selective
advantages arising from the combination of factors embedded in the facilities of
the human body: high mobility as provided by the walking apparatus, the
respiratory and cooling system, manipulation of environmental conditions through
action of the hands, superior communication through language, orientation
through the sensory system, and coordination of all facilities by the brain, and
in the last 50,000 years, the evolution of the symbolic instrumentarium.
(Anthro, (Bednarik
1994), (Calvin 1991), (Goppold
1999d), (Kingdon 1997), (Leroi-Gourhan 1984), (Semiotica 1994), Skoyles).
According to Calvin (1991), key factors in the evolution of intelligence were
the superior anthropoid facilities of spatio-temporal orientation, coordination,
and action. In the millions of years of struggle for hominid survival, a
decisive influence in shaping the neuronal infrastructure that made us human,
was the ability to throw objects with precision at moving targets. On the
neuro-computational level, this translates into an immense neuronal
infrastructure for processing the necessary information for orientation,
self-stabilization, target-tracking, and trajectory projection. Here, all the
specific human body facilities come into play: the upright posture, with freely
mobile upper body portion useable for supporting and enhancing the lever action
of the arms (one only needs to watch a modern diskus thrower for this), and the
spatial orientation system of the eyes, coordinated by the command system of the
brain. In those millions of years of hominid pre-history and R&D, the
arsenal of paleolithic weapons and tools was shaped: traps, snares, boomerangs,
propulsors, atl-atls, bolas, slingshots, axes, and (fire-) drills. (Bellier
1990). A problem with the archeological accounting of these effects is that the
durable remains of ancient technology, the stones, tell less than half the
story. Stone in itself is extremely inflexible, and cumbersome to use, and only
by mounting stones in handles and shafts, can suitable tools be fashioned.
Therefore, it is only through practical experimentation and re-engineering that
one can gain an insight into paleolithic ingenuity (Anthro). By the intensive
use of seemingly primitive, but very effective, technology humans had
transformed large regions of the planet Earth already 10,000 years ago: by using
fire, their tools and weapons, and the most devastating device of them all: the
human population bomb (aka the baby boom). According to one hypothesis, the
extinction of the large mammalian pleistocene fauna on all continents except
Africa is the work of paleolithic hunters. (Kingdon 1997).

In this long pre-history, the requirements of force-coupling
ballistical and mechanical devices to a still evolving human body provided the
co-evolutionary stimuli for shaping the neuronal infrastructures that supported
the later facilities of symbolization. The subtle connection can be found if we
consider the etymology of symbolon, stemming from sym-ballein: "to
throw together", whose aspect of throwing reappears in the word
ballistics.

11.4.2. The era of high perfection of
Neuronal Resonance Technology

and its
decline

The high art of tool and weapons design in the pre-industrial
age consisted of close-fitting the weight-hardness-elasticity-impact
characteristics of any tool to the corresponding biorhythmics of the moving
human body. In the terminology of current neurological research, this art of
tool design was aimed at perfecting the Neuronal Resonance (abbrev. NR) feedback
loops between the tool and the human body. Its key ingredient is rhythm, applied
as productive factor in the coupling of extrasomatic implements (tools, weapons)
to the human body. In the following context, the keyword for this will be:
Neuronal Resonance Technology (NRT). Rhythm is the temporal patterning of
events, thus NRT is temporal technology. The "secret" of the Egyptian Pyramid
building technology was the applied science of human mass mobilization and
precision coordination through rhythm. Equally cogent examples of routine
application of this skill can be found in all the megalithic monuments world
wide.

The perfection of neuronal resonance effects was the essence
of all the master craft traditions of humanity, and following the
industrialization, the principles of NR design were largely forgotten in modern
civilizations (Bernard 1985, Bücher 1924, Giesecke 1991, Goppold 1999d,
Morris 1986). There are only some notable remnants left in the field of sports
gear, like skiing, tennis, or golf equipment. Also, Japan represents the lone
exception of a highly industrialized civilization that preserves its heritage of
master craftsmen as "national treasures". The wholesale loss of the "secrets" of
NR design can be attributed to a compounding of several social, economic, and
technological factors. With the advent of power machinery, the essential
necessity for designing the tools to fit with the human body-rhythmics, to make
them operable efficiently at all, fell away. The social factors of NR design are
that the highest evolution of artisanship depended on the close client-patron
relations of a mature and sophisticated feudalistic society, which supplied the
expert user community, and the necessary financial backing to evolve skills that
needed years, and even decades of apprenticeship on the side of the artisans. A
notable case of such a co-evolution of skills and expert users is again provided
by Japan, which had possibly the most evolved feudalistic system of all human
societies, and managed to preserve its structures intact until the mid-1800's.
Here, the examples of superior and unmatched skills abound, like the legendary
swordsmiths, and the equally legendary users of these products, the Samurai. As
the Samurai case shows, the essential ingredient of pushing the limits of NRT is
the co-evolution of the human-technology dyadic system, with the tools
exquisitly matched by the craftsman to the specifics of that particular human
body. Apart from the extremely high cost of such development, which prohibited
its use on a larger scale, the Samurai example also shows the ingredient of
secrecy that was part and parcel of this societal arrangement. Even if it were
possible (and this is doubtful) that such skills could efficiently be taught to
mass audiences, this was not the aim: such swordsmanship was "too good for the
masses", and the military superiority of the Samurai depended on their
finely-honed reflexes (aka neuronal resonances), an edge which they liked to
maintain over the much less-well trained common soldiers. (A vivid demonstration
of this is found in the movie "The Seven Samurai"). In Europe, the mechanization
of warfare set in with guns and mass-drilled, mass-operating armies, and
displaced the high-skilled aristocratic individual warrior. But the connection
with the aristocracy is still evident in the sports scene, where the highly
skilled use of exquisite weapons was supplanted by an equally skilled use of
sportsgear whose technical perfection was honed to the sole aim of superior NR
for its user.

Another factor contributing to the loss of NR awareness in
current technology is to be found in a philosophical orientation of the Western
intellectual elites that dates back at least 2500 years. In the so-called
philosopher wars between pre-Socratic and pre-Platonic Greek schools, the
proponents of the Eleatic school of Parmenides and Zeno, held the view that
motion and change were impossible and non-existent, as evidenced by the famous
Zeno paradoxes. Against them, the proponents of Heraklitean views argued that
everything was change and motion: "panta rhei". In the further course of events,
the views of the Eleatic school became dominant, as expressed by the idea system
of Platon, and furthermore, in the domination of eternal essences in the
Christian religion. (Gadamer 1989, Goppold 1999d, Heidegger, Heraklit 1976,
Heuser 1992, Lippe 1997, Parmenides 1974, Platon 1988, Pleger 1991). In the
further development of Western sciences, especially under the lead of
equilibrium physics, time is "geometrized", it is "exorcised from the system of
natural laws", as Gotthard Günther (1976, x) describes it, similarly Straub
(1990), and Whitehead (1969). According to Halang (1992), computer science also
suffers from an underdeveloped awareness of temporality.

The Renaissance saw the breakup of the medieval craft guilds
and unions and the release of their closely-guarded trade secrets, and the
printing revolution brought a transfer of this knowledge into book form. This
put a selective filter on those forms of knowledge which were amenable to the
print medium, and it hindered the transmission of all those kinds of processual
knowledge which could not (easily) be written down. (Bernard 1985, Bücher
1924, Eisenstein 1979, Giesecke 1991, Goppold 1999d, Morris 1986). NRT is in
many respects antithetical to bookish learning, and when this form become the
dominant civilatory knowledge transmission system in Western societies, it
caused a decline of the awareness and usage of NRT. Ludwig Klages (1981), in:
"Der Geist als Widersacher der Seele" has drawn up a life's work in the grand
scenario of a historical struggle of social coalition groups and human
intelligence types, which could in contemporary terminology be called a silent
war, and a vonNeuman type coalition game, of "Formal Symbolic Intelligence"
against "Neuronal Intelligence". Similar statements are made by Mumford (1934,
1977), and other workers in cultural anthropology, kinesics and related fields:
(Birdwhistell (1970), Chernoff (1994), DerraDeMoroda (1982), Diamond (1976),
Franko (1993), Hanna (1979), Jeschke (1983), Lamb (1979), Levi-Strauss (1978),
Lippe (1997), Moore (1988), Spencer (1985)).

The demise of aristocratic rule, the once-stronghold of NRT
expertise, also brought the end of its societal application, with the exception
of fringe fields, like sports and of course, the military. The new ruler class
of the bourgeois Western societies, the law-makers, law-enforcers, and
law-interpreters, the bureaucratic civil servants, and the merchants, derived
their education mostly from the letter oriented public school system, and made
it through its upper echelons, the equally letter oriented universities.
(Creveld 1999).

11.4.3. Neuronal Resonance Technology and
the Military

The secrecy argument and the "too good for the masses" cause
of NRT holds especially true for military applications. The same NRT skills that
enabled the ancient Egyptians to coordinate thousands of workers for their
pyramid constructions were also employed concurrently with the first mass armies
of history. From then on, the military remains strongly conscious of time
factors and a strong user of NRT. Military drill, mass formations and
operations, are an essence of military operation. For the temporal coordination
of the huge human masses, music and rhythm was the prime tool. (DeLanda 1991,
Ferrill 1985, Foucault 1969, O'Connell 1989). History abounds with examples of
successful application of music as weapon, from the sounds of the trumpets that
shattered the walls of Jericho, to the Janissary music, accompanying the Osman
conquests that almost trampled European civilization underfoot. With the
mechanization of warfare, the driving beat of military operations was hammered
by the sound of machinery, and in WWI, the Scottish bagpipes only led the
soldiers into the stakkato of the machine-gun bloodbath. The Roman Legionaries
had probably reached the ultimate achievement in mass NRT applications. Arab and
Mongol cavalry represents perhaps the outstanding peak for the NRT application
of coupling human and animal bodies into a whole, and uniting thousands of them
into a steel-fanged blizzard. A horrible fascination perfuses the
masterpieces of NRT application throughout history, for their appalling
effectiveness, and utter defiance of anything resembling human values and
emotions. Nomen est omen: the Fasces, the old Roman Symbol of unification
of many people under one order and will, became the hallmark of Faschism.
Their resourgence and renaissance was in Hitler's Blitzkrieg formations of
motorized and mechanized warriors mounted on tanks and airplanes. Essentially,
the present-day cyber-age war scenarios have driven NRT to unprecedented
extremes, with human nervous systems burrowed further and further down into
ever-evolving electronic sensory-effectory-stimulus-response feedback loops. We
are therefore not surprised to find the most advanced NRT cybernetics in present
high-grade weapons systems: aimed to perform essentially the same purposes as a
million years ago, but with "a bigger bang for the buck". Time remains the
essential factor: the winner is the one who shoots first, and most accurately.
The importance of time factors in military application can be diagrammed with
two "magic" triangles: The triangle of: Mobility-Security-Firepower, and the
triangle of the OODA-Loop.

The "magic" triangle of:
Mobility-Security-Firepower

Ferrill (1985: 44) describes "the three indispensable
ingredients of warfare": mobility - security - firepower. The diagram positions
mobility in the apex of the triangle - above security and firepower. This
serves to indicate its vital importance and the necessity for emphasis. Probably
more battles were lost due to neglect of the mobility factor than for any other
cause. Its paramount influence is evidenced by the two greatest defeats that
almost stopped European civilization in its tracks: The Arab and the Mongol
conquests, both of which were due to neglect of the mobility factor. The heavily
armored, cumbersome, and immobile European knight cavalry proved ineffective
against their swift-moving enemies. If mobility is retained intact, then evasive
action even after a lost battle can ensure survival and ordered retreat of the
main body of forces. The three factors stand in a tri-polar tradeoff relation,
and finding the optimal balance between them is the acme of weapons systems
design, today as well as 5000 years ago.

While the "Mobility - Security - Firepower" triangle
represents the basics, or the "trivium" of warfare, the essential element of the
high "Art of War" (Sun Tsu) is the information and intelligence factor.
This is introduced with the OODA Loop: Observation - Orientation - Decision -
Action (Stein 1998). It is the guiding rule of today's "Information War"
scenarios.

The "magic" triangle of the OODA-Loop: "Observation,
Orientation, Decision, Action"

Turned the other way, NRT is "too good to leave it to the
military", and it is far too dangerous to have them alone control it. Let us
re-trace the connection between symbolon, syn-ballization and
ballistics. The archaic Greek roots preserve the traces of an otherwise
long-lost connection. We re-consider the spatio-temporal feats involved in the
neuronal infrastructure processing of throwing: Throwing a stone at a moving
target (which chimpanzees can manage, badly), and the technologically advanced
feat of throwing an atl-atl, or still further down the evolutionary line, the
shooting of an arrow (which re-appears as mouse cursor on all computer screens).
These actions involve Spatio-Temporal Perspective, another word for the
orientation and ordering systems embedded in our neuronal control programs. The
arrow, this ubiquitous pointing symbol, is for us, an unconscious carrier of the
million-year old control programs of our neuronal systems, which embeds the
primordial operation of symballization: the throwing together of a
diverse set of stimuli into one object, which can be pointed at, shot at
(ob-jectum), and thus brought into our possession, physical or mental.
Here lies the connection of the symbolizing done with words and graphs, to the
archaic roots. "Perspective" as used here, is a visual and
spatial, as well as temporal, principle, that evolved through a
million-years honing process of hunting and being hunted. The drawings at
Altamira, Chauvet, and Lascaux prove the perspective ability of paleolithic
hunters, not by the modern Renaissance usage of the word, but by even deeper,
more fundamental abilities: to literally "ban" the forms and features of the
creatures they depicted, onto the rocky, uneven surfaces, using the features of
the rock to enhance the visual effect of their paintings. The kind of
Perspective that was achieved in the Renaissance, is only a formal
hierarchical method of depiction that is based on those millennia-old roots. In
the present context, the word Spatio-Temporal Perspective will extend
this further, to mean a method for the hierarchical ordering schemata in
conceptual organization. The following diagram shows how the scientific ordering
schemata, which derive from the ancient Arbor Porphyricus method, are
descendants of the primordial Spatio-Temporal Perspective.

Symbolization and Perspective as hierarchical ordering
schema in conceptual organization

11.4.5. The Pyramid of Spatio-Temporal
Perspective

This figure can be merged with the two "magic" triangles of
the military application that where presented above:
Mobility-Security-Firepower, and the OODA-Loop:
Observation-Orientation-Decision-Action. This can be visualized with a Pyramid.
Viewed from one aspect, we see the counterbalancing "mobility - security -
firepower" tradeoffs, and viewed from the other aspect, we have the mutually
reinforcing poles of the OODA loop.

On the reigning apex of the Pyramid, we now find the factors
of time, and of Observation and Orientation which we can
also call Information. Thus the picture reveals another vista of the
Perspective, because we can turn this picture by 90 degrees, so that the
eye is now on the top. In German, the focal moment of the Now is aptly
called "der Augen-Blick".

11.4.6. The Eye on the Pyramid as Symbol
of Temporality and Orientation

With this series of morphings, we have now arrived at that
famous symbolon which appears on every US One-Dollar bill: The Eye on the
Pyramid. It is interpreted here in a different meaning than what the Freemason
influenced US founding fathers had in mind. The letters mark the essential
elements of human temporal orientation:

A: The Present - The "Now" - The Cogent Moment

The Present is the focus of all existence. We cannot
act and think but in the Now, and also Memory, the mental
projections of a Past, and Expectation, the projections of a
Future, can only happen in the present moment. In German, the Now
is called "der Augen-Blick", which again leads us back to the old
symbolism. In neurological terms, the Now is governed by a temporal
coherence function spanning about three seconds: "the three second
consciousness" (Pöppel).

C: The Future: The Expectation

B: The Past

B1: Personal Memory

B2: Collective / Cultural Memory

B3: Phylogenetic Memory, Genetic Heritage,
Instincts

D: The Forgetting, Death, Dissolution of Memory

11.4.7. The tradeoff balance and the
potential of NRT optimized design

These temporal factors mark the tradeoff balance and the
potential of NRT optimized design in present-day multimedia information systems.
The design criteria are based on the close-coupling of the interplay between the
human cognitive and manipulative factors, the human memory, and the access and
display speed and data volume of the information devices. All these factors must
be brought into an optimal, and precarious balance, which is easily lost. The
military examples show that the optimal balance of
"Mobility-Security-Firepower", and the OODA-Loop:
"Observation-Orientation-Decision-Action" depends crucially on the technology
available, and on the aims to be fulfilled. The following discussion will
amplify on some of the relevant aspects in past, present, and future
UIT.

11.5. Factors of Space and Time in User Interface Technology

The current generation of user interfaces (GUI, aka WIMP) is
predominantly oriented towards the visual and spatial sections of the human
cognitive spectrum. It is the sensible course of action to use the available
technology to support the eye as the primary human sensory organ, and because of
a general visual and spatial preference of Western cultures. The diagrams above
have showed the role of the perspective metaphor that is available to integrate
the different, and diverse views offered by the information systems into a
perceptual whole. The work of Kim Veltman (1997, 1998) outlines a related
approach to achieve this aim. To further trace the deeper connections, effects,
and consequences of this predominantly visual orientation, one might apply the
media theoretical analyses of Innis and McLuhan to HCI paradigms in Kuhnian
sense, as well as the culture theoretical studies outlined in the above
paragraphs. (Cassirer 1954-1994, Chandler, Innis 1952-1991, Kuhn 1962, Marvin
1986).

11.5.1. Spatio-temporal integration as
"magic" factor in UIT

While visual and spatial factors command most of current HCI
R&D work, an overview of a sample of representative articles, like the CHI
conferences over the last years (ACM-CHI), shows that there is very little work
dealing with time factors explicitly, and even less work on the integration of
the spatial and temporal aspects. A notable exception is Tognazzini (1993),
which explicitly links the factor of time to "magic", and describes the working
methods of stage magicians as "manipulations of time" (p. 359). It has been
noted in the section above ("The era of high perfection of Neuronal Resonance
Technology and its decline"), that there are deeply rooted cultural and
philosophical factors why time considerations tend to vanish from view of
technological and economic agendas.

11.5.2. The Human / Computer
Bottleneck

The most serious bottleneck separating the diverse worlds of
human and computer information processing is the peephole display capacity
available with present technology and the narrow bandwith of user inputs
available. This is especially serious when compared to the vastly increased
power of other computing technology over the last 50 years. This shows the
immense disparity of an approximate 1000-fold increase in RAM and HD capacity
and similar increase in CPU processor power with the almost standstill in
display and input technology.

The present GUI / Mouse technology may make things easier for
users, but it causes a bottleneck of ist own: the WIMP bottleneck. The
older technology of keyboard access is (at least for touch typists) about 10
times faster than mouse clicking. This is compounded by more adverse speed
factors of the WIMP paradigm and the loss of what little haptic and kinesthetic
somatic interaction potential the keyboard had offered in the older generation
of UIT. See the section further down: "Time and Ergonomic Penalties of Present
GUI UIT".

Diagram of the Human / Computer Bottleneck

11.5.3. The SW industry: an inversion of
Moore's law

A cursory inspection of the dynamics of SW systems development
in the PC industry gives the picture of an inversion of Moore's law: It is easy
to get the impression that negative time factors are a prime marketing strategy
of the industry. Even while hardware performance is continually rising, the
practical effect is that the newest software seems to be purposefully designed
to use up computing resources to such an extent that systems are so slow to be
practically useless when they are not run on the latest and most powerful
hardware on the market.

A closer examination of industry system design policy shows
the enormous attraction of GUI design and comparative neglect of time factors:
The mass market of naive, unsophisticated computer users that was opened with
the Macintosh has turned out to set a standard that was rigidly adhered to, even
where the design limitations of this 1984 machine were since long broken by
present hardware power (Common 1993). This translates again into a Kuhnian
paradigm, whose stability is not determined by what is technically possible, or
rationally advisable, but by social standards of the "least common denominator"
and "no experiments, please" flavor. The situation is that of a vonNeumann
player coalition between a monopolizing SW industry and the "lowest common
denominator" standard of the largest-possible user base. As there is a huge mass
market, there is also a tremendous inertia, and no industrial player wants to
play the guinea pig trying out any new ideas and approaches, especially not the
largest one, who seems to have opted for the technically most inferior
possibility (Businessweek). Thus, there has been little progress beyond the
basic design decisions of the Macintosh. But, as everyone notices, these "tried
and trusted" paradigms seem to be wearing out (Landauer, Norman), the market
saturation has peaked, as present systems seem to be headed towards a continuous
asymptotic approximation of Dinosaur efficiency. The salient point behind this
is the necessity to keep the upgrade cycles of hardware and software continually
turning, while the quality of the systems seems to be on the decline.
Compounded, this development leads into a Lemming effect, where "more and more
of the same" is produced, and very little thought is given to the interplay
factors between the technology and the human organism - the theme of NRT
outlined above.

11.5.5. Result of the VonNeuman Game for
UIT:

The current Von Neuman Game of player coalitions results in
this situation:

maximally protecting their trade secrets, keeping
user-relevant system information and modification potential locked up in
proprietary and encrypted formats.

in coalition with:

2) A vast, unsophisticated User Community who is served by
WIMP UIT

Least common denominator translates into largest customer
base.

Locked out are: Power users.

Features lost: User Configuration, Local Modification, Local
Adaptation, Transparency of SW.

Exceptions / Resistors of the general trend are:

Linux, Free SW Foundation, GNU, Open Source, BeOS

11.5.6. The Inverted Triangle of:
Firepower - Security - Mobility

The current VonNeuman Game has the following consequence for
UIT. An inverted "magic" triangle of: Firepower - Security -
Mobility. The Mobility (speed) factor is sacrificed to
unilaterally increase of Firepower and Security, both of which serve best the
interests of the present coalition members.

Diagram of the Inverted Triangle of: Firepower - Security -
Mobility

11.5.7. If transportation technology had
kept the same pace as computers...

A well-known sales-hype on the state of affairs in computerdom
is based on a comparison between computers and cars, and it goes about like
this:

"If transportation technology had kept the
same pace as computers, a Rolls Royce would cost about one Dollar and would go
about thousand miles per gallon of fuel."

But this is an euphemism or simply a
make-believe. A more correct statement would be:

"If transportation technology had run the
same course as computers, we would all be using battle cruisers or armored
transports and tanks, for driving to work, to the supermarket, or to the
disco."

Closer scrutiny of the design principles of
current computer systems yields a classic case of obliteration of the mobility
factor in the basic "mobility-security-firepower" triangle of the military that
was presented above. The classic design aim of current systems is
Firepower: "to get a bigger bang for the buck", indicating the featuritis
that is the current rage of the industry. The issue of Security is mainly
centered around protecting the interests of the SW vendor's copyrights. The
dominant Industry-Political Security Factor is: "Don't Sell Winchesters
to Indians". It doesn't increase the profits of the SW if the Users become too
powerful or too productive.

With present system failure levels of the Win-XX standard,
Securityfor the User seems to be as low as any time.

11.5.8. Time Optimization in Earlier
Generation Mini Computers

Time factors were of prime importance in the earlier mini
computer generation. Again in Kuhnian sense, there was a complete reversal of
paradigms between mini computers, early microcomputers, and the present
generation of Macintosh-Style PCs. In the older paradigm of minis, design
constraints were imposed by the coupling of a fast hard disk with relatively
small computer power (PDP type), which forced the systems designers to
painstakingly optimize systems performance around that combination, resulting in
such unrepeated feats of temporal efficiency as APL, and MUMPS. These were
undoubtedly the most powerful programming languages ever invented by man, and
just in terms of pure efficiency, present systems are a big step backwards. But
these systems were also cryptic and unforgiving, hard to train, and hard to
maintain, and so there were good reasons for the paradigm switch. And, of
course, it generates more business volume, when millions of users are catered,
than a few thousand.

We come back to the magic: The "olden days" of computing were
an age of wizards, and the connection with Tognazzini's article, the "magic
factor" gives an indication of a time efficiency that cannot be re-gained by
predominantly a visual / spatial oriented framework, once it has given up
control of the time factor. One typical effect of the GUI mouse access is, that
it slows the user down by about a factor of ten, compared to the very rapid
keyboard input of hotkeys (or command line sequences). Of course to be that
fast, the user has to have memorized all the command key sequences, and must be
a touch-typist. With the complexity of today's menu interfaces, it is impossible
to learn them all (especially when every vendor uses different hotkeys for
equivalent functions, or, as Microsoft does, creates a new assignment with every
new release). As the APL solution shows us, there is a different strategy
possible, to contain menu depth and complexity by combinatorial strategies, but
this is not easy to learn.

11.5.9. Time Factors, Flow, and the
"Augmentation of Human Intellect"

For the "Augmentation of Human Intellect" (Engelbart), the
time factor seems to be crucial. Unfortunately, the "magic effect" also causes
problems with verification and rationalization by academic standards. This may
be a reason why Engelbart has spent a lifetime churning out ideas of which the
largest portion still remains to be recognized, let alone be put into wide
usage. As was already pointed out, time is a blind spot of computer science (as
additional material, one may read Feyerabend for underlying reasons why this may
be so). In the literature, there is a body of work around "Flow"
(Csikszentmihalyi, Karn 1997: 64). This is a somewhat loose term for
hard-to-define intellect-augmentation effects that can occur, when expert work
is able to proceed in uninterrupted sequences of cumulative efficiency. In this,
the time factor is critical, since it is interrelated with the human attention
span and capacity of the short term memory, the best known of which in the
'flicker fusion effect' utilized in movie projection (Pöppel 1978-1995).
Maximum time lag of about 1/10 sec seems imperative. Noticeable augmentation
effects are attained mainly when a high level of user training and expertise is
started with at the beginning. Unfortunately, since the "expert" or "power user"
population, that would benefit most from this research, is much smaller than the
"general user" population that is catered for by the common GUI interface, the
financial rewards for such R&D are too small for large-scale industry
efforts. Progress will likely depend on individual efforts, like Engelbart's. A
"royal road" to travel would be a combination of efforts: presenting an
easy-to-use GUI interface that is accessible to the widest possible user
population, coupled with an industry standardized end-user programming facility
(EUPL), and a configurable user interface language (UIL). In the accompanying
paper (Goppold 1999h), a time oriented HCI design is presented, that seeks to
regain the efficiency factors of the former computing generations, but is based
on a widely available platform, the Java VM, and extensively uses the hypertext
principle to make it accessible for large user populations.

11.5.10. The Need for the Integration of
Hand and Mind

As was pointed out above (Bernard 1985, Goppold 1999d,
Leroi-Gourhan 1984), human intelligence is not a factor of pure visual
(Platonic, and derived therefrom, Christian) cognition and mental operation (the
Logocentrism), but is based on the vital modes of a human body operating in the
flesh, and depends most vitally on the combination of hand and mind. The almost
exclusive concentration on visual stimuli, which is a heritage from this
Platonic-Christian abhorrence of all things fleshly, and a total dedication to
the pure spirit (of the visual mind), brings a disruption of the
coordination and coherence of the body-mind whole, which is the essence of
embodied intelligence.

The former generation keyboard interface allowed wider
utilization of the potential of the hand. The mouse turns this expressive
potential into a single-channel serial device, thereby seriously crippling it.
In this aspect, the mouse transforms the HCI into a model closely resembling the
Turing Machine, the ultimate serial device. The incongruency is, that the human
brain is a massively parallel device, and the human body is able to perform a
multitude of actions in parallel. Even touch-typing, though not really a
parallel operation, gives a greatly enhanced spectrum of manual operation, by
the mutual synchronous temporal interplay of both hands. The long term effect of
exclusive WIMP use possibly leads to a degeneration of human potential because
there are no provisions any more for improving one's effectiveness, like
touch-typing was. The subtle interconnections of hand and mind have been lost
with WIMP UIT.

11.5.11. Need for UIT supporting the
somatic potential of the body: Haptic and Kinesthetic Sensory Modalities

The Graphical UIT appeals mainly to the eye, and supports
people whose main abilities lie in this domain of mental facilities, while the
keyboard is more suitable for touch oriented people (applies especially to touch
typists). Those people were systematically disadvantaged by the switch to WIMP
UIT. There is a lot of talk about more modern multi-sensory UIT about to be
developed, like the much-touted work of Hiroshi Ishii, at MIT (Ishii 1999a,
1999b), but the inherent potential of the keyboard has been completely
neglected. We are forgetting that this machinery is quite well adapted to the
most versatile human expressive device, a technology that is also cheaper than
almost anything else and that could still be improved quite a lot.

11.6. Time and Ergonomic Penalties of Present GUI UIT

The next sections will give a short overview of some of the
most problematic ergonomic dead-end roads into which the present GUI SW paradigm
has led. The popular graphical UIT (WIMP) is certainly an advance in terms of
broad market acceptance and -pentration over the old command line oriented
interfaces accessible only by keyboard. But this mass market appeal of current
GUI applications like the WWW browsers also sets the UIT standard to the lowest
common denominator of the naive user whose preferred working attitude is
serendipity browsing. What is almost completely lost in these solutions are the
upgrade options to switch to more powerful modes of HCI, even if these are
somewhat more demanding in training, thinking, and memorization on the side of
the user. The exclusive, and alternative-less unconditional universal adaptation
of the GUI paradigm in present computing systems reminds us much of the
proverbial fairy tale involving the granting of a wish that has a hidden
insidious drawback that comes to the attention only much too late when nothing
can be done any more to offset the damage. Underlying is a neglect of
information ecology, a factor which can only partly be covered under the
heading of ergonomics.

There are tradeoffs which turn the ease of use for naive users
into a dead end road when it comes to blocking the speed potential for the
expert user, especially through rapid hotkey entry, and shell and macro
programming. This fault is not necessarily inherent in the GUI technology itself
but is a result of throwing out backwards compatibility. Current software
systems are in many cases not completely operable by keyboard any more. A good
case in point are WWW browsers, which don't allow the cursor control via
keyboard, so that it is impossible to access the html screen clicking areas this
way.

11.6.1. The WIMP UIT time
penalty

GUI WIMP interfaces are useable by the most unsophisticated
user, but they impose a heavy time and nervous stress penalty on the expert
user. The daily routine of using WIMP software involves delving through hundreds
of menus always pointing and clicking mostly to the same spots on the screen in
the same programs. Each point and click may use up maybe a second, and it is
about ten times slower than a keyboard hit. Accumulated over grand use time in
one year, this adds up to hundreds of wasted hours which could be saved by using
hotkeys, shortcuts, and macros, at the small expense of some training and
memorization. Making a ballbark estimate of about 100 working hours cumulatively
spent pointing and clicking per year, multiply this by about one million power
experts world wide, by about $ 250,-/hr compound cost for highly professional
work, and one can easily sum it all up to a figure of about $ 25 billion worth
of expert user time and capacity lost each year to inefficient UIT. And these
are the most knowledgeable experts who would have ten times as much business, if
they had the lifetime to do it. It is of course in the best interest of the
computer industry (and the rest of us) to cater for those hundreds of millions
of users who will never want to become power users, and provide an easy-for-all
UIT, but is the social price to be paid also accounted for correctly? It is not
only the power users, the highest paid professionals, but a path for increasing
effectiveness, that is barred by the WIMP UIT. This cannot be shrugged off so
lightly.

11.6.2. The WIMP RSI
syndrome

What is harder to account is the nervous stress factor
involved in the point-and-click orgies of current WIMP mazes, here called the
WIMP RSI. The featuritis that is the current rage of the update-driven profit
making machinery of the SW industry forces them to increase the depth and
complexity of menus with every new release, and often relocating menu positions
to completely different places in the tree. And one will never find provisions
for users of the old software versions to get back the old menu layout schemes,
to which they were used and trained. This stress factor imposes on the expert
users a totally different type of RSI, than that which affects typical clerical
keyboard users.

Every professional expert WIMP user can attest to the syndrom
of WIMP RSI him/herself: There is a constantly recurring (depending on task,
about every ten seconds) need to take the eyes and the attention off the parts
of the screen where the work information resides (we may call this the work
focus area), and then search for and focus on the control areas,
fiddling and fingering around on the table until one has finally found the
mouse, and then engaging in a pinball-wizard like game trying to fine-position
the cursor on those tiny control areas on the screen euphemistically labeled
scroll bars, activation buttons, and what-not. This necessity to constantly
shift visual as well as mental attention and focus is the most problematic
aspect of the WIMP UIT. In the long run, this permanent attention interrupt
turns WIMP buttons into a nerve-consuming hurdle race. The nervous damage shows
very indirectly through such symptoms as cumulative exhaustion (computerese:
burn out), and by its psycho-social after-effects.

Repeatedly shifting eye and attention focus selectively
disadvantages older people. Nervous RSI hits the harder the higher up we go in
age, and on the expert level of the user. And it is probably easy to see, what
is a boon for the one (naive users, young people), is a bane for the other
(experts, older people). And the most critical resource in today's information
age is the time and nervous energy of the professional information worker. And
one of the essentials of the human condition that don't change so much, even
over the millennia, is that older people are more experienced than younger ones,
accustomed to deal with a much deeper memory structure, even while their
exterior sensory facilities are not as acute or as fast as those of young
people. The ergonomic advantage of WIMP as mass selling argument for the novice
"computer kid" population has turned into an information ecology trap for
the expert users and those whose main ability lies in their high experience
potential.

The computer industry cherishes an unspoken, but rarely
critically examined, myth that just because some gadgets are newer, they must
necessarily be better than the old methods and tools. This is perfectly fine as
far as the marketing of new gizmos by the industry is concerned, but it is not
always up to the truth and the best interest of the users. The much maligned
keyboard interface is a prime candidate for re-valuation of its ergonomic
advantages. We only need to re-christen it for its potential as the most
evolved synchronous, bi-manual HCI technology. Its history traces back to
that famous musical instrument of western tradition: the piano. This was also an
NRT application of much greater sophistication than the typewriter keyboard,
since it allows simultaneous operation with both hands, even input with the
feet, and the keys are analog-coded, and it gives immediate audible
feedback.

11.7.1. Speed

Key access is about ten times faster than WIMP. If one has
taken the small training expenditure to memorize the location of the keys, one
will never have to search long for any key. One hundred years of experience by
millions of touch typists all over the world provide a strong argument that only
the WIMP driven computer industry seems to ignore completely. A contributing
factor for this ignorance seems to be that the touch-typists were usually of the
lower-paid clerical low-prestige echelon, and they were women, and so their
experience belongs to a lesson society-at-large has never learned, especially
those company CEOs and company stock holders who thought it to be beneath their
dignity to learn this essential skill, and who always used their workers for
this demeaning lowly task.

11.7.2. Positioning Fidelity and Less
Mind Load

The keys are always in the same place relative to each other,
and relative to the whole apparatus of the keyboard. This translates into faster
access speed and less searching for the activation buttons. Since one can easily
memorize the location of the keys, one will be able to activate them completely
mechanically, without thinking or taking your attention off the more important
work. We will never have to bother about the program control, once we have
mastered the essential hotkeys. This is the essential ergonomic asset that was
wiped out by WIMP SW, which mostly dosn't offer hotkeys any more these days.

There are as yet unrealized ergonomic potentials in the
keyboard that would increase its usability immensely: One is the possibility of
braille-coding all keys. Not only would every computer user in the world be able
to read braille by this, giving an additional societal advantage for blind
people, which they really need. But the immensely more consequential advantage
is that the training of the keys would be about ten times faster. Why? Because
we would instantly have the tactile feedback from the fingers themselves, on
which key they had just landed. The main problem with training touch-typists is
that the ordinary typewriter-keyboard offers only visual feedback and guidance.
This again necessitates the untrained user to constantly look at the keys, to
see where the fingers have landed. The hard part in training touch-typists is to
get them off the habit of looking at the keyboard. But if keys were braille
coded, the keyboard would be practically self-teaching, and everyone could learn
it within a day or so. One may speculate about the reason why no-one has thought
of this: western societies are overdominantly visually oriented and have
relegated the tactile domain to fringe areas, like blind people. One could also
say: western societies are touch-blind. And the WIMP UIT is a symptom of that
societal neglect. It has focused entirely on the visual domain throwing out
almost any tactile element. Implementing this idea would mean multi-billions of
dollars of saved user time all over the world, at practically no additional
cost, but it would also obviate the need for a lot of today's WIMP technology.
And the industrial economic marketing argument of WIMP is that it hogs computer
resources, and thus keeps the business of the ever-increasing upgrade loop
turning.

11.7.4. Combinatorial
Richness

Combined with ALT CTRL SHIFT, it offers about 400 different
codes one can enter, each within a mean access time of about 1/10 sec.

11.7.5. Visibility

The keyboard is big enough that it doesn't get easily lost in
a maze of papers on a table top. You never have to search for it, and if you do,
it is probably better that you get yourself a check-up with your friendly local
shrink.

11.7.6. The compound potential of
keyboard and shell interface in the UNIX system

In the present view, the script-programming factor of the
keyboard interface is the most important feature that has become lost in WIMP
interfaces to the disadvantage of expert information workers. The
programmability was a commonplace feature of the the earlier
Pre-XwindowsUNIX system. It derived its power
from the philosophy that all i/o was essentially a file, no matter where it came
from. So it didn't matter if the input came from the keyboard or from a disk
file. That made UNIX shell programming an expert's dreamland of cryptic command
code adventures. Before windows programming, any application program could be
shell-scripted as well as commanded from the keyboard. That facility seems to
have been largely lost in the industry.

11.7.7. The battle between the mouse and
the turtle: Systematic disadvantagement of keyboard users

One can even speak of a systematic disadvantagement of
keyboard users. It is possible to move the mouse pointer everywhere on the
screen with the mouse, but the industry seems to have obliterated the keyboard
activated cursor instead. Everyone can try this with the standard WWW browsers
which don't even supply a cursor any more. Why is it not possible to move the
mouse pointer with the keyboard also? The industry has invented scores of
devices to move the mouse pointer around, like trackballs, touch pads, and what
not, but it has "forgotten" to make it moveable by the keyboard. Behind this
seems to loom an unexamined "battle of paradigms" (in the Kuhnian sense). The
main advantage of the keyboard to position the mouse pointer is that one doesn't
have to move the attention as well as the fingers off the keyboard too much. The
problem seems to have been one of sloppily designing distinctions into i/o modes
that are purely historical. The mouse input can move the cursor everywhere on
the screen, whereas the keyboard cursor control keys are either inoperatives or
they can only move in limited areas. The underlying issue is a question of
program control philosophy which we could call "the battle between the mouse and
the turtle". WIMP UIT is dominated by a design philosophy of central command and
control in the cartesian manner, and as indicated, it is essentially visual. On
the other hand, the keyboard controlled system means (user- / position- /
program state-) relative command and control, mainly a tactile metaphor. The
tactile element involves also that one goes from here to there by n steps in the
direction +-:angle. This has been made the central systems philosophy of the
LOGO programming system. It was immensely popular and successful for teaching
programming to small children, but the deeper issues seem to have been lost to
the computer industry. This has to do with local and ad-hoc authority, local
autonomy, and local decision. In some way, this local autonomy spirit was part
of the "hacker ethics" which by now seems to have been carefully purged from the
industry, and would there not be such a strong support for systems like LINUX
and the GNU group, it would have disappeared altogether. Characterized by the
visible images of a Turtle and a Mouse battling each other, these are two
entirely opposite programming and control and command metaphors which don't
easily mix, and are hard to bring together in SW program control. But it is not
impossible. And this should be an aim for UI engineering.