A Call for Collaborative Interfaces

Abstract:
In this note, I call for a move towards viewing interfaces as means
for people and computers to collaborate on solving problems rather
than means for people to control computers. This collaborative
perspective on user interfaces can apply quite
broadly, and not only provides a source for novel interface techniques
but serves as a good tool for analyzing existing interfaces. The view
affects thinking on interfaces primarily by motivating a different
split in the roles and responsibilities of the two participants in
problem-solving, the computer and the user.

In this note, I call for a move towards viewing interfaces as means
for people and computers to collaborate on solving problems rather
than means for people to control computers. This collaborative
perspective on user interfaces [1] can apply quite
broadly, and not only provides a source for novel interface techniques
but serves as a good tool for analyzing existing interfaces. The view
affects thinking on interfaces primarily by motivating a different
split in the roles and responsibilities of the two participants in
problem-solving, the computer and the user.

Many (though not all) computer-based activities can be thought of in
terms of optimally solving some problem of interest to the user --
writing a (maximally) convincing memo, determining the (ideal) price
for a product, constructing a (maximally) communicative diagram.
Viewed as optimization tasks, such problems are typically hard, either
in the informal sense of the word (it's hard to be convincing, for
instance) or in the technical sense (the optimal price may be
specified by a nonlinear program) or both. As a concrete example,
consider the problem of constructing a diagram that communicates well
some information in the form of a network diagram of nodes and links.
This problem is hard in both senses, in that the aesthetic criteria
are subtle, and combinatorial idealizations of the problem, such as
graph layout with minimum edge crossings, have been shown to be
NP-hard.

Computers are unlikely to be of much use in solving an informally
or combinatorially difficult problem, so the standard paradigm is for
the user to solve the problem and the computer to record the solution.
One might use a word-processor to record the memo, a spreadsheet to
record the computation of the price, or a drawing tool to record the
diagram. But such an approach gives up far too easily; though the
computer is unlikely to be of aid in solving the entire problem, the
user's efforts can be leveraged by splitting the duties between the
computer and user more equally leading to a more collaborative style
of interaction. In particular, computers can be quite good at local
optimization, whereas the user's mental effort is best expended on the
global aspects of the problem.

Under this view, a key problem in user interface design is in defining
a representation of the problem to be solved such that it can be
naturally decomposed into a local portion that a computer would be
good at and a global portion left to the user. In the case of the
network-diagram layout example, computers are quite good at
fine-grained placement of graphical objects whereas a user may find
pixel-accurate placement, requiring fine-grained mouse gestures,
extremely tedious. The user's role is more appropriately the
coarse-grained layout, which is in fact the combinatorially more
difficult part. Thus, an interface device that turns fine-grained
into coarse-grained gestures, such as a snap-to-grid feature,
exemplifies the collaborative interface view expounded here. Of
course, using more information about the task can allow for even more
of the burden to be placed on the computer. For instance, in a
network diagram layout task, nodes are typically placed in standard
positions relative to other nodes -- horizontally, vertically,
diagonally, or radially aligned. An interface might leverage this
fact by providing a snap-to-point feature where the points are defined
not by a fixed grid but by the particulars of the current layout.
Such devices are now seen in some drawing and design packages.

Our own work on collaborative interfaces for network diagram design
has attempted to take this perspective even further, allowing the user
to specify that nodes should be placed in certain configurations
typical of network diagrams. The computer is used to locally satisfy
these constraints while the user simultaneously structures the global
layout. Both participants are acting at the same time to further the
goals specified by the user, with the user doing the deep,
combinatorial part and the computer the fine-grained, pixel-level
placement. Thus, by adding even more semantic information about the
task, an interface allowing even more equal collaboration can be
built.

In general, this approach to interface design can be realized by
viewing the task as a combinatorial optimization problem, and handing
the local optimization part to the computer while the user performs
the global part. This typically provides a way of using to best
advantage the abilities of computer and user. The key to designing
an interface then becomes representing the problem in such a way that
this nice division of roles is feasible.

Note that using the computer for performing local optimization leads
to a much different spin on optimization technology. For most other
applications, one wants the optimization method most likely to produce
the global optimum. Here, by contrast, one wants a method that is as
predictable in its performance as possible, even at the expense of
global optimality. Since we have already split up the optimization
responsibility between the two parties, it is imperative that the user
be able to intuit well how his or her actions will impinge on the
solution under construction. For this reason, we have concentrated on
using optimization techniques based (loosely) on simulating physical
systems; users can have strong intuitions about how the interface will
then behave based on analogy with the physical world.

This view of the desirability of fine-grained collaboration already
underlies many good user interface devices and designs: they make use
of task-specific information to leverage the actions of users,
performing fine-grained details while leaving the high-level aspects
to the user specifiable through gross interface gestures. But making
the view explicit means that we can look for more opportunities for
such collaborative interfaces, and push the ideas further, leading to
problem-solving software tools that better leverage human performance
through simple yet sophisticated user interfaces.

Acknowledgments

Joe Marks contributed to the formulation of this position statement.
I am indebted to him and Kathy Ryall for discussion of the topics
described herein.

The term "collaborative interface" as used here is intended to
emphasize collaboration between user and computer, not collaboration
among users, as in research on computer-supported cooperative work
(CSCW).