Generating Visual Arguments: a Media-independent Approach

Nancy Green, Stephan Kerpedjiev, Steven F. Roth

School of Computer Science

Carnegie Mellon University

Pittsburgh, PA 15231 USA

ngreen,kerpedji,roth@cs.cmu.edu

Giuseppe Carenini, Johanna Moore

Intelligent Systems Program

University of Pittsburgh

Pittsburgh, PA 15260 USA

carenini,jmoore@cs.pitt.edu

Introduction

The research reported here is part of our ongoing effort
(Kerpedjiev et al. 1997b; 1997a; Green, Carenini,
& Moore 1998; Green et al. 1998; Kerpedjiev et al.
1998) to design systems that can automatically generate
integrated text and information graphics presentations
of complex, quantitative data. In this paper,
we take the position that certain types of arguments
that can be presented visually in information graphics
(e.g., bar charts and scatter plots) can be gener
ated from an underlying media-independent represen
tation of a presentation. In support of this claim, first we briefly describe the architecture we are developing
for the generation of integrated text and information
graphics presentations. In this architecture, media-
independent communicative acts are transformed into
user task specifications which are the basis for the automatic
design of the presentation's graphics. Then we
present an example showing correspondences between
the media-independent representation of an argument
and the tasks that would be used to design a graphic
expressing the argument.

In our hybrid approach, shown in Figure 1, the con
tent and organization of a presentation is rst planned
at a media-independent level using a hierarchical plan
ner (Young 1994), resulting in a presentation plan. The
presentation plan describes the intentional and infor
mational structure of the presentation (Moser, Moore,
& Glendening 1995; Moore & Pollack 1992), as well
as what low-level media-independent communicative
acts are to be performed by the system to achieve the
presentation's goals. A media allocation component
decides which parts of the presentation plan to realize
in which media. Two media-specic generators (text,
graphics) then realize their assigned parts of the plan.

The text generator converts its assigned part of
the plan to functional descriptions of sentential units,
which are subsequently realized by a general-purpose
sentence generator (FUF/SURGE) (Elhadad & Robin
1996). (The complex process of converting the plan
to text is beyond the scope of this paper.) Graph
ics generation is performed in two stages. First, the
graphics generator converts the parts of the plan as
signed to it by the media-allocation component to a se
quence of user tasks that will enable the presentation's
goals to be achieved. (Previous integrated text and
graphic generation systems, e.g., (Andre & Rist 1994;
Fasciano & Lapalme 1996; Feiner & McKeown 1991;
Maybury 1991; McKeown et al. 1992; Wahlster et
al. 1993) have not attempted to automatically de
rive user tasks from a media-independent presenta
tion plan.) The task sequence is then input to the
SAGE graphic design system (Roth & Mattis 1990;
Roth et al. 1994), which automatically creates a
graphic designed to enable the user to perform these
tasks. All design decisions are made by SAGE, from
the type of graphic (e.g., a bar chart), to specic prop
erties of a graphic (e.g., the choice of horizontal as
opposed to vertical bars). In this way, graphic de
sign is tailored to a presentation's goals. (For details
on our approach to graphics generation, including the
derivation of tasks in our system, see (Kerpedjiev et
al. 1998).)

Expressing an Argument in Graphics

In this section, we give an analysis of an argument and
its representation in a media-independent presentation
plan. We describe the user tasks which would be de
rived from the media-independent communicative acts
of the plan in our current approach, and then suggest
some ways in which the structure of the discourse may
also contribute to the design of effective graphics, as
well as its in uence on media allocation.

Figure 1:Integrated Text-Graphics Generation Architecture

Analysis of example

The data used in this paper is fictitious.

The goal of the example presentation is for the user
to accept the belief that a certain local newspaper,
the Post-Gazette (PPG), has more readers than the
total number of readers of all other newspapers that
are subscribed to in some region. The user is currently
ignorant of this fact, but probably would not accept it
just on the basis of a simple assertion by the system,
due to his current beliefs. In particular, the user knows
that the New York Times (NYT) has more readers
than the Wall Street Journal (WSJ) in the region, and
erroneously believes that because of this, the New York
Times must have the largest number of readers of all
newspapers in the region. However, the latter belief is
incompatible with the belief which it is the goal of the
presentation to get the user to accept.

To simplify discussion, let us abbreviate the propositions playing a role in the example as follows:

Q: the number of readers of NYT exceeds the number of readers of WSJ

R: the number of readers of NYT exceeds the number of readers of any other paper in the region

T: the number of readers of PPG exceeds the total number of readers of all other papers in the region.

In summary, the intentional structure of this argument can
be represented in the plan shown in Figure 2.
(The figure shows only the hierarchical relations among
the communicative acts and the core-contributor distinction
among acts; unlabelled acts are contributors.)
Such a plan might be realized in text as follows: Although
the New York Times is read by more people in
Western PA than the Wall Street Journal, the New
York Times does not have the highest number of readers in
the region. The Post-Gazette has more readers
than the total number of readers of all other newspapers
in the five-county Western PA region.

Realization in Graphics

A graphic realizing this argument is shown in Figure 3.
The core of the argument, the assertion that T
holds, is expressed by enabling the user to perform the
task of comparing the upper bar's length (which represents
the number of readers of PPG) to the lower bar's
length (which represents the total number of readers of
the other newspapers). (In general, an assertion that
some quantity is greater than another quantity would
be transformed by our graphics generator into a comparison task;
for details on the process of deriving a
task sequence, see (Kerpedjiev et al. 1998).)

The contributor given in support of T is expressed in
the same graphic, although less prominently. Its core
consists of the assertion (not R), which is expressed in
the graphic indirectly by falsifying R. That is, by enabling
the user to perform the task of comparing the
length of the segment of the lower bar labelled NYT
to the length of the upper bar, the user can see that
there is one newspaper (PPG) with more readers than
NYT, which falsifies R. The concession Q (contributing
to the acceptance of the assertion that R does not
hold), is expressed within the lower bar by enabling
the user to perform the task of comparing the length
of the segment labelled NYT (representing the number
of NYT readers) to the length of the segment labelled
WSJ (representing the number of WSJ readers).

Figure 3:Graphic realizing the argument

Thus, in our current approach each the three low-
level communicative acts of this plan would be tansformed
into the comparison tasks described above. The
tasks would then be used by SAGE to design a graphic
such as the one shown in Figure 3 to support these
tasks. Note that more than one graphic may be designed
by SAGE to support the tasks. An interesting
open question that we are investigating is how the
intentional structure of an argument should influence
graphic design. In this graphic, for example, the assertion
corresponding to the core of the argument is
more visually prominent than the other information
since the graphic contains only two horizontal bars, one
for each of the entities compared in the core assertion.
Also, the quantities compared in the core assertion are
encoded differently (i.e. as horizontal bars) from the
other quantities (i.e. as segments of a stacked bar).
The user may interpret the difference as conversation
ally implicating an important distinction in the two
sets of quantities (Marks & Reiter 1990).

A related issue is the role of discourse structure in
media allocation. For example, if the graphic shown in
Figure 3 is accompanied only by text realizing the core
of the argument (e.g., The Post-Gazette has more readers
than the total number of readers of all other news
papers in the five-county Western PA region), then the
text would contribute to the user's recognition of the
main point of the graphic. On the other hand, if the
same graphic is accompanied only by text realizing one
of the other acts of the plan (e.g., The New York Times
has more readers than the Wall Street Journal), then
the text might impede the user's recognition of the
main point of the graphic. In future work, we hope to
address these open issues.