Who can you trust?

FEATUREThe ambient mirror

Authors:
Dimitris Grammenos

Ambient intelligence envisions a future in which our
environment is populated by an infinite number of interoperating,
computing-embedded devices of different sizes and capabilities
[1], which are interweaved into “the fabric of
everyday life” and are indistinguishable from it
[2]. The ultimate goal of all these devices is to
serve human needs through the provision of a wide range of
physical, digital, and hybrid services [3] that
improve the quality of life by making it easier (smart homes,
e-commerce), safer (accident prevention and avoidance, monitoring
the location and safety of children), healthier (assisted living,
telemedicine), more productive and efficient (teleworking,
traffic management, driver assistance), and even more pleasant
(social-interaction and entertainment activities).

The first step toward meeting any human need is, of course,
identifying it. Ambient intelligence technologies achieve this
task through a vast collection of hardware and software modules,
which can generically be described as “sensors”
[4], since, in one way or another, they have the
ability to collect information that is, implicitly or explicitly,
produced by humans. These various pieces of information can then,
individually or collectively, be mapped to human actions, states,
intentions, and, eventually, needs.

Examples of hardware sensors include simple mechanical or
electronic devices, such as microphones, video cameras, distance,
movement and pressure sensors, as well as more sophisticated
apparatuses such as electroencephalographical devices and neural
implants. In general, all these sensors detect physical “output
signals” of the human body, whether sounds and gestures,
physiological measurements, exerted forces, electrical signals,
or brainwaves.

Software sensors complement the hardware ones but do not have
a physical embodiment. Their role is to detect the immaterial
(i.e., intellectual) products of human activity created through
the mediation/support of information and communication
technologies. For example, these sensors may monitor
Internet-based services such as email, Web portals, chat, and
search engines, but also operation systems or typical desktop
applications, like word processors, spreadsheets, and computer
games.

The information gathered by the various sensors is then
propagated to software modules thatassuming the role of
miniature brainsstore it, analyze it, change their own
internal status and then decide accordingly upon related actions
that should be taken, often affecting the state of the user’s
physical or digital environment. In a future
ambient-intelligence-enhanced world, there may be several such
decision-making modules of different sizes, capabilities, and
sophistication working independently, collectively, or even
antagonistically.

The Ambient Mirror

Generally, unlike the approach of Big Brother, described in
George Orwell’s 1984, it is expected that in the emerging
ambient intelligence environments, most of the collected
information will be distributed among the numerous sensors and
applications (i.e., many “small brothers”), never making it to a
centralized repository of any kind; also, due mainly to storage
constraints, a large part of it will never be permanently
stored.

But what if, at some point in the future when practical
barriers are alleviated, the option of accumulating each and
every piece of information, no matter how trivial or elaborate,
related to the whole life of a single human being becomes
feasible? Would we then be able to re-create a complete and
accurate representation of that personnot only her
appearance, actions, interests, and habits but also her
personality and way of thinking?

And what if, instead of feeding this model into software
applications and services, we presented it to its rightful human
owner? In this case, the biggest obstacle that we would probably
stumble upon would be that of data rendering, or how to present
such massive and diverse, and potentially chaotic, data in a
structured, meaningful, and comprehensible way.

For the sake of our hypothesis, let’s suppose that we
eventually manage to create an appropriate rendering machine,
which we entitle the Ambient Mirrorthe contents of which
are accurate, noncontroversial, and human-readable.

Then, the first thing that one would probably ask is, “Will
the person in question be able to recognize their actual self in
this digital reflection?” You see, humans tend not to keep a
“high resolution” account of who we are, what we think, or what
we do. Instead, we draw a rough sketch of ourselves based on a
small, and sometimes fictitious, portion of the available
information. Our brain constantly filters millions of details,
retaining only a few important facts or events, often creating
idealized, mitigated, or aggravated versions of them.
Furthermore, over time, much of this retained data is forgotten,
corrupted, pushed back, or merged.

If we somehow manage to get a satisfying answer to the
aforementioned question, then the next step would be to identify
possible “applications” of such a tool and assess their potential
usefulness, as well as their impact both to the individual and
the to the society as a whole.

Personal Use of the Ambient Mirror

On an abstract level, the Ambient Mirror can be described as a
tool for recording past personal experiences. Over the course of
a few thousand years, humans devised and used several other tools
for the same cause. Indisputably, the first one was memory. Then
came the spoken language, writing, as well as art, mainly in the
form of painting and sculpture. In the modern age, the task is
further facilitated through a multitude of electronic devices
such as microphones, photographic/video cameras, and computers.
Of course, all these “traditional” means will also be basic
constituents of the Ambient Mirror, but in a novel, pervasive
(and probably miniaturized) manifestation, as well as integrated
with additional monitoring technologies and reasoning components
that will extend their grasp and complement their abilities.
Table 1 illustrates an attempt to sum up the
foreseen differences between past practices and the Ambient
Mirror.

Based on its unique characteristics, some of the possible
personal uses of the Ambient Mirror include:

Extending human memory and awareness. For
anything ranging from today’s schedule or self-medication plan,
to vacation memoirs or critical incidents of one’s life. In this
case, the mirror works like a recording/ playback device that can
present information to its owner both (pro)actively, based on
predefined programs (“always remind me about a meeting an hour
ahead,” “when I meet someone, remind me of her name”) and
passively, using dynamic requests (“I want to re-experience: (a)
X specific minutes of yesterday; (b) all the times that I felt
happy during the past month; (c) a collection of my personal
thoughts expressed today”). Since the mirror collects data from
far more sources than the human senses, it can potentially
“remember” more things about the self than this person can even
perceive, thus not only increasing the storage capacity of the
human brain, but also (retrospectively) enhancing its awareness
capabilities.

But if we rely even more on technology for remembering things,
would we then deliberately further weaken our own memory? Could
we arrive to the point where we use our brain only for short-term
storage of data needed for immediate processing and then dump
everything else on the Ambient Mirror, in a way that’s analogous
to RAM and hard disks? And what will happen if the mirror
“breaks,” for example, it is partially destroyed or is infected
by a virus? On the other hand, if the mirror helps us free all
the memory space and processing overhead devoted to mundane
things, would we then be able to put these resources to better
use? Beyond these concerns, another question is, would the
capability of accurately replaying past events result in an
endless torment of regret about things that we should (or should
not) have said and done?

Seeing the unseen. In other words, converting
abstract personal information into a tangible form. The mirror
intrinsically possesses this ability since most of the collected
data is imperceptible to human senses (physiological measures,
analysis of written and oral communications). This information
can work as real-time feedback for various uses such as stress
control, accident prediction and avoidance, or instant health or
mental status checkup before an important match, performance, or
meeting. A related example is that of biofeedback, in which
bodily functions (blood pressure, heart rate, skin temperature)
are measured and then visualized (using numbers, graphs, or even
games) in order to help a person understand and master her
unconscious physiological activities. Furthermore, the mirror has
the ability to keep track of, spot, and report phenomena that are
usually difficult to self-identify because they span or change
over time, such as habits, interpersonal relationships, likes,
dislikes, acquirement of (or loss) of skills, degradation of
senses, etc.

Self-knowledge and self-improvement. Development
of physical skills (much like what athletes and actors do by
reviewing videos of their performance to identify errors and
correct their technique) and introspection/self-reflection for
scientific, educational, and psychological purposes. A basic
concern of this type of personal use is that currently some
guidance or monitoring by a trained specialist (a trainer or a
psychologist, depending on the case) is required in order to
ensure the safety of the person in question and the achievement
of tangible results. In this case, one may claim that required
expert knowledge can also be offered by the Ambient Mirror,
either in the form of intelligent agents or by acting as a
mediator between the person and remote experts. Another potential
risk is that the mirror may work as a self-fulfilling prophecy;
the person looking into it may (sub) consciously alter their
actions or even character traits in order to “align” themselves
to their perceived image.

Social Use of the Ambient Mirror

If, as admitted earlier on, we can really see ourselves in the
Ambient Mirror, then the mirror probably would also be a handy
tool for allowing other people to “see” us. Currently, this
function is served by personal conversations, profiles, CVs,
biographies, blogs, photo albums, and video clips. The key
difference with the mirror is that it will not just allow you to
see somebody else, but to almost experience being somebody
else!

Of course, in this case, one would like to be able to decide
who has access to which parts of one’s mirror. In other words,
some kind of “tools” will be needed for selecting specific parts
of the mirror’s content and releasing them to selected viewers.
An ethical question raised here is related to the fact that, as a
person’s life path (intentionally or accidentally) intersects at
some point in time with that of other people, the mirror will
also contain data referring to them, compromising their privacy.
A possible way to overcome this problem is through a pervasive
filtering mechanism, which will be able to anonymize, block, or
even erase data recorded by all or selected other Ambient
Mirrors. Such a solution may create new problems, since one can
foresee that this function can easily be exploited for
“stealthing” malevolent actions. Additionally, if someone managed
to get control over other people’s filtering mechanisms, they
could apply a novel form of censorship that can reach to the
extent of virtually erasing all evidence of a person’s
existence.

The related potential applications of the Ambient Mirror
include all those situations in which personal information needs
to be disclosed, such as interpersonal relations and
communication, work applications, and medical filing, among
others. Furthermore, the mirror could advance the art of
autobiography to a new level, since it will allow the “reader” to
almost relive the “writer’s” life, experiencing a kind of “living
reincarnation.”

But what if the mirror is also used as a piece of evidence,
like in a courtroom? Would anyone be able to use fragments of
their mirror as a valid testimony? Is there a possibility that
someone manipulates the content of your (or someone else’s)
mirror for someone else’s favor? Would a third party (the state,
the police) be allowed access to anyone’s mirror, thus
materializing part of the Big Brother scenario?

And, since, in our example, the mirrorup to this
pointis considered a personal artifact, whose interests
should it serve when a conflict arises? Its owner’s or those of
the “universal” truth? In other words, would my mirror lie for
me? Would it give me away to my boss when I ask for a day off
pretending to be sick or to my wife when she asks me where I was
last night? And what about all these cases of “white lies”
imposed by political correctness, for example, when one asks us
if we really liked the food, the dress, the painting, or if we
had a good time? Would the mirror become a tool of unprecedented
rudeness?

Afterword

In thenot so distantfuture, it is quite possible
that ambient intelligence technologies will provide scientists
with all the components and knowledge required to build a device
similar to what is described in this article as an Ambient
Mirror, reflecting every trace of human existence. Ideally, this
mirror will become a new means of altering human self-perception,
as well as that of others, eventually leading to autognosis while
also fostering mutual respect and understanding. Furthermore, the
mirror may even develop into a kind of “synthetic conscience,”
shepherding people when they go astray. Then again, if the mirror
is not working or properly used, it could totally distort our
view of the world with unpredictable, but unquestionably
catastrophic, consequences. Furthermore, the mirror might prove
to be the ultimate surveillance mechanism, which would
permanently end the notion of privacy as we know it.
Nevertheless, as is the case with any other type of technology,
the Ambient Mirror cannot be characterized, per se, as good or
bad. Its use by our society will provide the final verdict.

In his afterword to Elhacedor (“The Maker”), Jorge Luis
Borges wrote: “A man sets out to draw the world. As the years go
by, he peoples a space with images of provinces, kingdoms,
mountains, bays, ships, islands, fishes, rooms, instruments,
stars, horses, and individuals. A short time before he dies, he
discovers that the patient labyrinth of lines traces the
lineaments of his own face [5].”

So, if in Borges’s passage the man discovers in his drawing
just “the lineaments of his own face,” what would the same man
discover by looking into the Ambient Mirror? The lineaments of
his soul?

Dimitris Grammenos is the lead interaction designer of the
human computer interaction laboratory of ICS-FORTH, where he
heads the lab’s universally accessible games activity. He holds a
B.Sc. in computer science and an M.Sc. and Ph.D. in electronic
engineering. Grammenos has expertise on interaction design issues
in traditional Windows-based environments, as well as in
technologically advanced environments, such as virtual and
augmented environments, wearable computers, and ambient
intelligence environments. He has been involved in several
European R&D projects related to design for all and universal
access, and has given related lectures, seminars, and tutorials.
For more information visit:
http://www.ics.forth.gr/hci/people/dgrammenos.html.

Permission to make digital or hard copies of all or part of
this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or
commercial advantage and that copies bear this notice and the
full citation on the first page. To copy otherwise, to republish,
to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.