The Elements of Interaction Design

“For interaction designers,
who create products and services that can be digital (software)
or analog (a karaoke machine) or both (a mobile phone), the
design elements are more conceptual.”

Other design disciplines use raw materials. Communication designers
use basic visual elements such as the line. Industrial designers
work with simple 3D shapes such as the cube, the sphere, and
the cylinder. For interaction designers, who create products
and services that can be digital (software) or analog (a karaoke
machine) or both (a mobile phone), the design elements are
more conceptual. And yet they offer a powerful set of components
for interaction designers to bring to bear on their projects.

Motion

In much the same way that inert gases don’t mingle
with other gases, objects that don’t move don’t
interact. An interaction is some sort of communication, and
communication is about movement: our vocal cords vibrating
as we speak, our hands and arms writing or typing as we send
email or instant messages, sound and data moving between two
entities.

“Interaction designers are
very concerned with behavior: the way that products behave
in response to the way that people behave.”

We communicate in many ways and through many different products,
from mobile phones to email. Those products and the people
who use them generate behavior, and interaction designers are
very concerned with behavior: the way that products behave
in response to the way that people behave. And all behavior
is, in fact, motion: motion colored by attitude, culture, personality,
and context. There’s wide variation even in such universal
and seemingly simple behaviors such as walking (that’s
why, for instance, there’s a need for both high-impact
walking shoes and walkers for the elderly), and the designs
we create have to understand and account for those variations
in motion. Even a simple motion like pressing a key on a keyboard
can be difficult if you are elderly or infirm.

Motion is often a trigger for action, as when your finger
clicks the button on your mouse. The triggered action (or at
least the feedback for that action) is often about motion as
well. You click a Web site link, and the page changes. You
press a key, and an email window closes. There is motion on
your screen.

Without motion, there can be no interaction.

Space

“Interaction designers work
in both 2D and 3D space, whether that space is a digital screen
or the analog, physical space we all inhabit.”

Movement, even on a subatomic level, happens in some sort
of space, even if the boundary of that space (as with, say,
the Internet) is unclear. Interaction designers work in both
2D and 3D space, whether that space is a digital screen or
the analog, physical space we all inhabit.

Most often, interaction design involves a combination of physical
and analog spaces. You make a gesture in physical, analog space—for
instance, turning a knob on your stereo—and you see the
results on its digital display screen. The reverse can, of
course, be true as well. You can stream music from your computer
through your stereo and into physical space.

Most interaction designers underutilize 3D space on screens. The
physical flatness of our monitors and display screens causes
us to ignore what the Renaissance painters discovered so long
ago: perspective. Objects, even in a 2D space, can appear to
move backward and forward in 3D space. Perspective creates,
alongside X (height) and Y (width), a Z (depth) axis on which
to work. Web sites are notably bad in their use of Z space.

Starbucks® cafes typically make excellent use of physical space, with
the ordering area separated from the fulfillment area where
customers receive their beverages and those areas separated
from the area where people can customize (add milk and sugar
and other condiments to) their drinks. Compare that to the
typical crush around a single counter of a fast food restaurant.

Space provides a context for motion. Is the action taking
place in a quiet office in front of a computer screen or in
a crowded, noisy airport in front of a kiosk?

All interactions take place in a space.

Time

“Interaction designers need
an awareness of time. Some tasks are complicated and take a
long time to complete—for instance, searching for and
buying a product.”

All interactions take place over time. Sometimes that time
can be near-instantaneous, like the time it takes to click
a mouse. Sometimes it can involve very long durations. You
can still find online usenet messages (usenet is a sort of
bulletin board system) from decades ago.

Movement through space takes time to accomplish. As every gamer will attest, it takes time to press buttons (around 8 milliseconds
at the fastest). Even with broadband speeds, it takes time
for packets of data to travel from distant servers through
the physical wires and perhaps through the air via wireless
signal to your computer.

Interaction designers need an awareness of time. Some tasks are complicated and take a long time to complete—for instance, searching
for and buying a product. Many e-commerce Web sites require
you to log in before purchasing, and that login session will
be active for a set time. Imagine if Amazon® or other e-commerce
sites timed out every few minutes and required you to log in
repeatedly while shopping—it’s unlikely you’d
buy much from them. Some travel and concert-ticket Web sites
make users race against the clock to enter their credit card
information before their selected seats are lost.

Digital time is definitely not human time. Digital time is measured
in milliseconds, a single one of which is considerably shorter
than the blink of an eye. Changes done by the computer can
be so instantaneous that programmers need to program in delays
so that humans can detect them.

You can feel the impact of milliseconds, however. Extra milliseconds
added to every keystroke or mouse-click would probably make
you think your computer is slow because of the tiny delay.
Several hundred milliseconds would cause frustration and anger,
and a single-second delay each time you pressed a key would
probably make your computer unusable.

Time creates rhythm. How fast something pops up on the screen or
how long it takes to complete an action like renewing your
driver’s license controls the rhythm of the interaction.
Games are often about rhythm: how many aliens come at you at
any given moment, or how long does it take to complete a level.
Rhythm is also an important component of animation: how quickly
does a folder open or close on the desktop, how slowly does
a drop-down menu slide open. Interaction designers control
this rhythm.

Battery life (the duration of which is slowly getting better) is another
element of time of which designers need to be cognizant. Some
features, such as a backlight, drain more battery power than
others and thus decrease the amount of time the device works.
A mobile phone that worked for only 10 minutes unplugged from
power wouldn’t be of much use.

Interactions happen over time.

Appearance

“How something looks gives
us cues as to how it behaves and how we should interact with
it.”

How something looks gives us cues as to how it behaves and
how we should interact with it. The size, shape, and even weight
of mobile devices let us know that they should be carried with
us. The sleek black or silver look of digital video recorders
like TiVo® devices tell us that they are pieces of electronic
equipment and belong alongside stereos and televisions.

Appearance is the major source (texture is the other) of what cognitive
psychologist James Gibson, in 1966, called affordances. Gibson
explored the concept more fully in his 1979 book The Ecological
Approach to Visual Perception, but it wasn’t until
Don Norman’s seminal book The Psychology of Everyday
Things, in 1988, that the term spread into design. An
affordance is a property, or multiple properties, of an object
that provides some indication of how to interact with that
object or with a feature on that object. A chair has an affordance
of sitting because of its shape. A button has an affordance
of pushing because of its shape and the way it moves (or seemingly
moves). The empty space in a cup is an affordance that tells
us we could fill the cup with liquid. An affordance (or, technically,
a perceived affordance) is contextual and cultural. You know
you can push a button because you’ve pushed one before.
On the other hand, a person who has never seen chopsticks would
be puzzled about what to do with them.

Except to the visually impaired (for whom texture often substitutes),
appearance also conveys emotional content. Is this product
whimsical or serious? Practical or playful? Appearance can
also convey other attributes that may be meaningful: Is the
object expensive or cheap? Complicated or simple? Daunting
or approachable? Single use or enduring? Structured or casual?

Appearance has many variables for designers to alter

proportion

structure

size

shape

weight

color (hue, value, saturation)

All of these characteristics (and more) add up to appearance,
and nearly every design has some sort of appearance, even
if that appearance is a simple command line.

Texture

“The sensation of an object
can provide clues as to how it is to be used….”

While texture can also be part of the appearance, how an
object feels in the hand can convey the same sort information
as appearance. Texture, too, can convey affordances.
The sensation of an object can provide clues as to how it is
to be used as well as when and where. Is it solid or flimsy?
Is it fragile or durable? Do the knobs spin or push or both?

Texture can convey emotion as well. A fuzzy plush object conveys a
different meaning than a hard metallic one.

Designers can also work with texture variables such as vibration and
heat to signify actions. A mobile phone can vibrate when a
new message arrives, and one could imagine it growing colder
the longer it’s been since a voice-mail message arrived.

Sound

“Sound is underutilized (some would say rightfully so) in interaction design, but even a little bit of sound can make a major difference in a product.”

Sound is a small part of most interaction designs, but it
can be an important part, especially for ambient devices and
alerts. Sounds possess many variables that can convey information
as well. You wouldn’t want a loud screech to come out
of your computer every time you received email, and a soft
whisper wouldn’t exactly cause traffic to move aside
for an ambulance.

Sounds are made up of three main components, all of which can be adjusted
by a designer:

pitch—How high in
range a sound is. Is it high pitched like a bird’s
song or deep like thunder?

volume—How loud a sound is.

timbre or tone quality—What
type of sound it is. Sounds played at the same volume and
pitch can seem very different. Think of a middle C played
on a trumpet and one played on a piano.

Sound is underutilized (some would say rightfully
so) in interaction design, but even a little bit of sound can
make a major difference in a product. Steve Jobs insisted that
the wheel on an iPod® make an audible click that could
be heard without headphones.

All of these elements of interaction design comprise any interaction
designer’s toolkit, and while interaction designers may
not consciously manipulate them, they are the building blocks
of interaction design.

17 Comments

They are the basic building blocks, the same way that a line is for graphic design. They may seem obvious, but one of the things missing in the discipline was a discussion of what gets manipulated by interaction designers to create their work. This was my answer.

I think that intentionally breaking things down to elements is not an obvious task at all. But the question you are probably thinking is why even bother? Is this going to help my practice?

It all depends on your style and interest. I can see something like this being used in a design studio curriculum where in a studio class these elements are explored as key ingredients. Just as by changing ingredients, heat levels, and length of cooking you can end up with completely different meals (let alone cooking surfaces).

I can see these elements being mixed and changed and skewed toward learning to master them as ingredients in a designer’s kitchen. This is a common practice with Graphic Design and Industrial Design where an entire studio might be about learning how color change alone effects the results of your design.

In my own day-to-day practice, I find that it is very important for me to intentionally imagine these or similar elements as I work on my designs.

Now some people may not be as analytical as myself and won’t get use out of this, but I do believe there are both academic and practical value propositions for looking at the above.

On a related, yet separate note, I wrote a piece expanding on where Dan left off from the above. I’ve been thinking a bit about “foundational” elements of IxD for quite some time, and I am happy that Dan is doing the same level of thinkig here. Here is my link:
http://synapticburn.com/comments.php?id=143010C

I agree with you about the underutilization of the Z-axis, which can be employed to show not only distance but time—for example, use perspective to show the user is starting a journey that will end at some point off in the distance.

Not a dimension, but perhaps an element: emotion. Also complexity. It’s interesting how these interact—for example it’s been shown that too many choices can result in a lessened feeling of well-being (and increased cognitive load). You could go on and on…

A nit or my mis-reading? :)
“Space” section, paragraph 2:
“Most often, interaction design involves a combination of physical and analog spaces. You make a gesture in physical, analog space—for instance, turning a knob on your stereo—and you see the results on its digital display screen.”
Should it be:
“Most often, interaction design involves a combination of physical and digital spaces.”
My reading of the space section equates physical space as analogous to the analog world.
Thanks, great article!

A gentle comment—After I read this, I wished I knew a little more about what the book would be about. The general concepts, or building blocks, of IxD are interesting, but I’d like to know more about how this book or these concepts could help me, specifically and pragmatically, with examples, in my day-to-day work as a designer. That may not have been the point of this article, but that’s what I was left feeling. Thanks!

Breaking them down is fine in a “Semiology of Graphics” sense. The interesting issues are determining the lexicon and how these elements interact. In the two dimensional sense, we need a Wolfgang Weigart of interactive. And, what about the most obvious of all: reaction?

The variables discussed are no different than film, with the exception of texture. It isn’t interactive unless an interaction is specified and acting with sound can induce texture. How is this determined?

BTW, an example of reaction or behavior would be Disneyland. As an interaction system it is top-notch. The behavior would be characterizations and elaborations such as cast interaction: happy. A result of elaborating on this, or creating rules, would be the cast manual. Do this… Don’t do this…

Wonderful discussion and article. I look forward to reading the book—reading it, not “scanning”. I agree with Roy Zornow’s comment about emotion and complexity. I’m certain this is covered in the book, but there is the issue of unconscious behavior and motivations to consider. Unfortunately, they happen to be among the most difficult to predict, quantify, chart, etcetera, because they are the elements of our humanity.

Who we are—the majority of us being people—seems as important as anything else. I’ve long believed that we are actually in the “people business” in addition to technology, design, etcetera. Understanding people belongs in that useful toolkit. Thanks for letting me share.

Michael, I do touch on both emotion and complexity in the book. Generating emotion through tools like mood boards and being aware of the context of use. Complexity is given a richer treatment, covering not only Tesler’s Law of the Conservation of Complexity, but also how to make things smart and clever by managing complexity.

Negative space is not a separate element, but a property of each element when no detectable properties exist. In most disciplines, this absence has become as important as the elements themselves. To graphic designers this is known as white space and is treated as delicately as the type they set. Just as space has negative space, motion has stillness, and sound has silence. It is with this absence of an element that we are able to create rhythm, balance, and hierarchy.

As a storyteller, I appreciate the detail around time and the multiple perspectives woven richly into this post.

My experience includes personas and scenarios that are different from most. I allow people to wear many persona masks in my analysis. This provides a simplicity and flexibility you may find helpful. Scenarios are often used and almost always abused by being treated as interactions or workflows—sensitivity analysis tools or worse—rather than a simple combination of unique stories. Accountants, oddly enough, get it right. Though they don’t recognize scenarios when they create forecasts. Consider that at the core of the numbers in a forecast is a bunch of stories. If you have two forecasts, generally it is much more than sensitivity analysis and can be described as two unique sets of stories in the same way as most strategic scenario analysts would work. Google Pierre Wack for scenario analysis history.

I also swap product for expertise when I combine personas and scenarios, in order to understand expertise gaps moving forward in time. In this case, I reverse the polarity by creating an ideal persona based on future products and the behaviors and stories served. Once this is done, I compare the persona to the expertise pool to determine the variances. This can be done across the value stream, either to support HR internally or competitive intelligence externally.