Lisbeth Rasmussen DDRE DK, Zack Jacobson CAC, CA

MartinTaylor, Martin Taylor Consulting, CA

Definition of visualisation
Evaluation for acquisition vs. eval for research vs. eval for iterative design
IST-05 Reference Model
Maps model to OODA loop
Six questions listed
Uses Santorini fresco picture as example of "visualisation system", goes through
and asks the 6 questions wrt this fresco
Some questions answered better than others (first three better than latter three)
Translucency
Foregrounding/backgrounding—less important objects placed higher up, further
back away from viewer
Fresco as interactive interface for "marine tasking order"
Click on elements of boat (captain, navigator) brings up relevant information
e.g., for captain, basic mission statement; navigator-route map, etc.
Don’t go to places set back, go to welcoming people, up royal road, to theatrical
area
Questions implied by questions ie.., Q1 becomes Q1a, Q1b, etc.
Eg.., what is user trying to achieve vs. Q1a who is the user?
In this case, captain, navigator, etc.

Q2 can user perceive there is progress
A2a, by scrolling whether ships have been assigned, departure and arrival locations,

ISSUE; How should the MTO Author be made aware if tasking has been completed.
Put an yellow or red x when task is incomplete
What affects the user’s ability to perceive
A4.1 fresco interface is physically long, so related items may not obviously
connect with each other
Solution: ensure that there are engines to test consistency among items such
as route map and text, captain’s timing orders
Red and yellow x’s provide alerting capability (Q6)
Word "interactively" in definition of visualisation ties in with most of the
questions

Zack Jacobson, CAC

Visual text mining for technology watch
Spot relations among elements of text in massive corpus—conceptual/meaningful
relations
Identify clusters of similar items, and scant conceptual regions—concepts not
represented in the data set
Search tasks: find the documents I need to read; find out where imp’t areas
of confluence are, and areas that should have confluence but do not
Vision allows us to choose those 3D elements of the scene relevant to the task,
e.g., driving
VITA visual interface for text analysis
Concepts identified by search engines top plane
Hits produced by search engines middle level
Bottom level is documents

Hits are small cylinders inside larger cylinder representing the document
Yellow spheres represent concepts
Artificial gravity moves private/sergeant away from other officer ranks
Second example shows that this approach not very scalable to large datasets

Tasks of TG on IF: Develop Conceptual Model of ASC, Develop and Evaluate Functional
Model of the ASC, Identify Functions that could be Automated
Operational Preparation of the Battlespace (OPB)
Physical, socio-political and operational environment lead to OPB, lead to C2
command support
Physical environment breakdown
Social political environment: cabinet, media, CIMIC, host cultural considerations,
allied and host political considerations
Operational environment G1, G2, etc.

ASC in Peace Support Ops
Military, disaster, socio-political, IS/COIN, and Crime Reports
Need to be classified, correlated, aggregated, and situation views created for
presentation to the commander

Linas Bukauskas, Aalborg U

Observer related data extraction
Show visible objects, show objects with specified visibility levels
Objects that will become (in) visible, objects that might be visible soon
Objects along a path

Create tree structure to access the data: tree orders objects according to
visibility factor
Also a second storage access tructure B-Tree
Visibility factor VF is function of two parameters object, and observer
Visible objects function of DB, Obs, and rho
Show me all items that have a visibility factor greater than rho (criterion)
But this is insufficient because it ignores importance/size of items

Leads to need to cluster, hierarchy
VR is minmal bounding squre (MBS), brightness color could be incorporated
In MBS objects are visible
Have hierarhcial structure of MBRs and MBSs
Group clusters of MBRs in hierarchical sequence

Eric Granum,

Demo in large circular display room, with stereo
Color data sets, scatterplots
Allows various Assignment of colors to conceptual dimensions
Can cycle through over time
Can see all as separate three way graphs in one space (lots of little 3 way
graphs)
Idea: integration of mapping issues with visual momentum ideas—need way to transition
between different coding approaches

Bill Wright, Visual Insights

CPOF: Observations regarding principles
Many visualizations over two years, lots of issues

Start with description of CPOF
Tailored visualizations: specific visualizations that align to user and to task
domain
Functional, Inventive, and Process Objectives
Get rid of icons, bring in blobology
Membranes—synthesis of aggregation and details
Provide an at-a-glance comprehensibility of force attributes
Lots of experimentation
Process: See users, do design, test design
SMEs are active participants and provide requirements
Blobs should have see, shoot, and sense attributes
Dynamic force strneght varies with density power contact
Blobs relative to function

do dynamic blos lead to greater understanding of important situation elements?
Do dynamic blobs help command discern patterns in the battlespace
Answer yes to both questions vs. control

Used simulation halt technique—importance of questions not assessed
Block Parties up to 1-6 Tactical Decision Games about 50

Force on Force Visualizations
Shows the CPOF displays
Bars used to represent attrition data
Incorporates brushing
Trying to show timeline data over geographic terrain, vectors fading over time
Circular blobs not working in urban terrain
Don’t get doubling of performance

details important to commander
Can we try to design with this in mind
Change blobs to suit what’s important at a given time
Got away from circles—circles were hiding important properties
Hiding relevant data
Outline/membrane approach—membrane being drawn around footprint of unit
Assumptions about data sources: radio frequency tags, digital compasses, tactical
inference engines

Individual symbols shown when zoomed in; when zoomed out get the membrane

Some principles of graphic design
Design is purposeful, for specific audiences

www.mundidesign.com
use of color perspective, layout

observations ‡ principles
use a methology
engage users
allow two-way learning: users and technologists
be aware
application of principles tricky
don’t use color
the more specific the visulization the more effective it is
let the data speak
intuitive means no legend
placeholders wdont’ work
rez,‡ can’t substitute with fake data
generate blobs from entity data
don’t use blobs for fewer entities

General Discussion

Wright’s list—use of color discussion
Jacobson: I like color, want to be able to use it
M. Taylor: color coding issues—don’t use color hue to indicate quantity, but hue
used to indicate a category (red forces vs. blue forces) is good
Color saturation can be used to indicate amount with some success, although
straight luminance/greyscale probably works just as well
Wright noted that use color as last resort

Jan Terje Bjorke: Discussion of Wright’s displays ­how memorable they were vs.
data shown in VR media lab
Hollands pointed out how the data types different

Wright made point about need for data to be realistic—accept no substitutes
V. Taylor: can’t always get real data

How to show uncertainty with blobs
Eric: Comparisons about visualizations
Augmented reality
Provides constraints, a formal language of a sort
Vs. complete freedom

Eric: Use of color—good for perceptual grouping
V. Taylor: need to know who users are

Try to be perfect: zack points out that can never do that
Wright says need principles, if you don’t get it perfect, people see the imperfection,
distracted by them
Emphasis on medium over message

Eric: set reference for level of quality you want
M. Taylor: calibrating level of quality with attention
DREO story about radar; stopped Canada’s involvement in holographic radar
Fresco e.g., highlighting important information

V. Taylor: different display formats, devices, PDA vs. large display
Must transmit as little as possible in small devices
Wright: generals with laptop

Smestad: what is meant by "at a glance" visualization
Relates to Ward Page attention comment
Wright: time to comprehend reduced when "at a glance"
Bad presentation systems, given enough time, can be interpreted

Varga: Wrightl’s presentation on tailored visualization; what should the rest
of us do
Wright says it’s diff between data viz vs. info viz.
Data viz has much less mapping between conceptual and perceptual dimensions

Jacobson: need incremental progress from one to another

Varga: don’t peak too early
Virtual reality as part of viz—using data and mapping it to time is one way
to go from data to information visualisatio

Bukauskas: what about air force?
Wright: Army aviation (helos) move like tanks
Bukauskas: would you use color for blobs?
Wright: Blobs not relevant, they are abstractions for hundreds of entities

V. Taylor: # of user interfaces and what each shows
Wright: interesting issue

Da Silva Verissimo: why did you switch representational scheme for Vita
Jacobson: wasn’t clear how things clustered with large datasets
V. Taylor: liked old version b/c could open up page easily
V. Taylor: wants feedback
How to select in 3D, likes idea of putting crown of thorns in Eric’s display

Thursday

Bill Wright

Topic: sonar tactical decision aid
Background, issues
Expected improvement in amount and quality of sonar data, and analytical models
ASW is a thinking war
Focus on one command level

Temperature in ocean varies in layers; temperature affects sonar
Sweet spots, shadows
Range on bearing—one dimensional in multidimensional world
Why not use isosurface—allows you to draw contours in a 3D volume
Got sketches from users

Key plan—incorporates traditional symbols: make new work with old
Show areas that have been searched
Add or manipulate assets and asset tracks
"what if" planning tool

thumbnails
save the result set—good for a period of time
model runs taken at different times
eg. At 40, 100, 300 m

kept existing way of doing things—original 2D graph—activity at one bearing,
different depths

what if playing with cutoff points on isovalues, isosurface
slider bars to change AN, FOM, POD—where you cutoff the values
ambient noise
FOM figure of merit
POD prob of detection
SV sound velocity

linked views—if changing depth in one view, other views should also be updated
if you play with your azimuth slice (bearing)
M. Taylor: how do you ensure that links are obvious
"make things live"
principle of dynamic query—ensure quick enough update rate
expose complexity of what operators dealing with to command level

Ross Walker/James Broad

Immersive Education Ltd.
Virtual Reality on Portable Devices
Large data on small amounts of silicon

Oxford U. Intel Education Initiative—why is software not doing a good job in
education
Take enjoyment of games, bring into education environment
Games use large amount of illusion with minimum of hardware

Millions of polygons
Game called "Republic"
Populated city environment
Newspaper found on street gets read, bolts on metal handrails realistic—progressively
more detailed info as zoom in, done with software
Integrated physics system
Set up role playing situations between characters and props
Dynamic lighting, shading

Discussion

Jacobson: do you have any potential applications (hammer without nails)—do you
have any nails
Walker: not from this group

V. Taylor/Wright: Millions of polygons—why no problem?
Amount of redundancy in data set
Broad: Simplicity of architectural forms vs. natural objects
M. Taylor: pre-existing polygons vs. creating them on the fly
Walker: Parthenon demo—took 6 man-months to create
(in response to Wright’s question: how long does it take to build these models)
model of athena inside, generally model quite realistic
shadows within objects but not across

To operators, engines operate on the raw sonar information
Engine algorithm understands current flows, temperature, salinity, etc.
Engines access historical database
Ray trace model
Output of model run compared to bathymetric data, etc.
M. Taylor: all these things in the dataspace
What info is sonar officer providing: probability or definitive
Active vs. passive sonar, deploy sonar or not, where to go, where to look
M makes analogy to chess player
Operator has to drive engines to look for patterns in oceans—shifts in arranged
data

Bjorke: probablility concept of response tied to application;
M: app is reason for doing this
Wright what is transformation—is transformation between situations between where
probability is relevant vs. not relevant
JT driving car 10 km/hr 10 m is long; but at 100 km/hr, 10 m is short
Different probabilities occur in each situation
M brakes on vs. not brakes on

Perceptual variables
Amount of information is limited by map users
In communicating map have grouped information into classes and by this reduced
amount of information
Scaling symbols with changes in map scale

Context determined by application area
Application of fuzzy logic
Use natural language sentences: e.g., distance from the airport to AOI is short
Apply membership functions 0-1 which describes probability statement is correct
Plots of membership functions for distance from airport, cheap hotel, etc.
Hollands wants all three—intersection operators from fuzzy logic—get a certain
set of hotels
If strongly recommend assign Yager operator (e.g., 0.8), then only one hotel
gets value greater than zero
Scatterplots are 2D maps

Iconic reps vs. linguistic sentences
Assignment of descriptors (small, large, very large) to graphical representations
divided bar—an iconic representation of magnitude
Distance between points can be used as an icon—scale the distances with respect
to frame zoom

Conclusions: do not overload user with info
Need for map design based on soft computing—fuzzy logic
Natural language sentences based on evaluations of quantiatitve data at nominal
level
General cartographic communication model—how to create communicating map
Excellent talk get this guy’s papers
Q&A: how to coalesce do not overload with give the users all the data
Wright: give them as much as possible with out overloading them; depends on user
Bjorke: Complexity comes from multiple views (over time)

Erik Kjems

VR for decision support in urban planning
Project at the VR Media Lab
Need to make model for visualiations so that ideas could be presented to town
council
Model: reusing texture frees up memory (reference to earlier presentation
contourlines,
road-project
orthophoto
billboards (trees, hedges, lights, signs)—only use two polygons, always pointing
against viewer
houses, photographed existing houses
other elements

primary req: as few polygons as possible, refresh rate at least 30 Hz
view should be as realistic as possible
consequences
use billboards, object splitting (split model into whole objects)—objects behind
not in CPU
as smooth as possible surfaces
photographic low res textures
VML model, plugin from maxim

Presentation in virtual envionment concave display
First minutes of silence when show to town council
A few insecure questions
Then heavy discussion

Removed buildings that were placed badly, had no view
Added green areas
New infrastructure
Increased distance between buildings
Decided to allow only one story housing

Survey
Have you seen the environment before—how high are your expectations to the presentation
Did the presentation clarify issues in the project Yes

Did the presentation influence the acceptance of the project
People who sought that presentation already wanted it , so no real change
How important was mobility and iteractivity in the model? Important
Mono/stereo—mono was good for town model
Was presentation worth the money--$6-7K
Texture model made in 5 days; texturing takes time 18 students 6 weeks

Primarily the modelling work, secondary the large display system
Interaction was very important

Second round
Discussions got much more into detail and even more intensive

Wright: Trees and houses separate data level? A Yes
Two source data sets height field and data overlay A multiple source, lots of
databases
Wright transportation, systems populations, A yes sure, would like to extend into
that area, but not right now
Wright could be useful for PDA
What kind of technical architecture—how to generate PDA material and deliver
it

Discussion

Jacobson: Spend time on Smestad’s sheet.Smestad’ places 13 guidelines on left
Visual summary Diagram on right
Uses the guidelines to assess one of Wright’s business diagrams floor/walls
stock information
Diagram supports many of points
Hollands pointed out coding schemes issues
Dynamic motion makes interpretation of different colors more obvious—some columns
hang down
Visual structure—how many units available at a time ­graph represents this

Captain Lars Abild Danish Army

DACCIS Presentation and Demonstration (Danish Army C2 Information System)
New Danish information system
Prototype system
Division breaking down into brigades
Danish army Almost entirely Danish division
NATO North East: Danish division with 12 PL div and 14 German div
Responsible for 710,000 km

5 projects within DACCIS
with each project are 4 phases
Operational User Group (OUG) set up especially for the DACCIS project, reporting
directly to the AOC (Army Operational Command)
AMC Army Material Command, Maersk Data, OUG are three parties involved in project
management
Brigade level—best guess organization 4 different HQs in brigade
not enough people for 24 hr period

DACCIS users should require less training than conventional system
Company educates instructors, instructors educate end users
Means end users don’t have to wait for next course at school

ATCCIS: working group create a data model, and replication model involving
many NATO countries incl. US, UK, CA
Too much engineers, not enough army

Data Model is ATCCIS General Hub 4 (for phase 2)

Replication model ATCCIS Replication Model
Denmark and Norway only countries with small contracts

Currently at phase 1

Data model includes relations between data;
visualization is a national implementation
Wright’s question: Expert Users changing every 2 years; what do you do to figure
it out
Answer: have to say OK, this was the best info we had, stick with original plan

Smestad: how many use cases?
A: about 200

Users want lots of map—most of DACCIS is map display
Scanned maps with scrolling and zoom
Move map with gesture
Select viewing vector by gesture (line of sight), get terrain profile, maintained
on map afterwards
Area of sight also possible—what you can see from that location
Phase 2 will incorporate 3D button
Zooming by selection of rectangular area by gesture
Can select particular features to be plotted in the maps using check boxes
Select objective area by selecting circle symbol as in drawing program, again
uses gesture to do this

Attack arrow—multi-headed possible based on user’s demand despite fact that
violates NATO standard
Number of heads indicates phase/order of attack (this group attacks first, that
second)
Select symbology from palette as in drawing tool

Interactive whiteboard part of system
Problem with shadows/blocking
Developed special purpose stick for reaching far side of display
Different levels of commander have different access to plan—commanding officer
can review/edit/approve plans of junior officer

Artillery position field of fire for different weapons done by selecting symbol
from palette; angle range chosen and is adjustable to produce pie slice

Palette choices affected by G2-G4 check boxes
APP6alpha (APP6A) same as mil std 2525
Right click on symbol, select history, shows track
Old problem of different locations on different officer’s maps
Shared displays—all information now common
Outlook used for mail
Gesture recognition/keyboard at whiteboard
Symbols need to be scaled with zoom

Margaret Varga, QinetiQ, UK

Going to talk about BDA and evaluation usabilty problems
CAOC wide
Air tasking order data in spreadsheet
Get summary of planned mission in bar chart
Problem bar chart not scalable
Table viewer
Field search
Also have problems

Discussion

V. Taylor: how do you get new technology to old spaces (people)
A: younger people more attuned to graphical representation
Gradual process of accommodation to new technologies

V. Taylor: research institutes looking at new technologies that may have application
down the road
Difficulty occurs when get genuine user, they have their own tunnel vision
A: have better contact with military community; how much can tolerate

Wright: acceptance; reach further into user; get better measures
A: difficult because have to have controlled environment
Relate graphics performance to operational performance; say to user "can only
get there if you can help us"
A: trying to do more trials, get deeper exposure to users; issues in getting
unclassified databases
User Interest in database visualization—fusions of database
Wright yes it’s big time investment
A: need for geography

Jacobson: database viz nice way to start
Pilots in combat like to use systems they trained on
Make new systems part of training (links to V. Taylor’s comments about military colleges)
A: yes, have to persuade the young one; but old one still has authority

M. Taylor: high and low ranking officers
Bringing them in at all levels helpful politically
A: Do compromise so that user can accept

Da Silva Verissimo: should try to get new features in

Smestad: creating guidelines for non-specific domains is a challenge; can you
explain
A: common solution is to have general set of tools within application; tries
to do much
Smestad: relate to my guidelines
A:
Smestad: graphics were problems, not guidelines

M. Taylor: need generics, but need to make sure not overgeneralizing across applications
Can have guidelines at various levels: hierarchy
Guidelines can be made to be compatible
This is always true, in this situation do this, in that situation, do that
Always refining guidelines
M. Taylor says Smestad’s guidelines are of the general form

M. Taylor: Smestad’s guidelines fit for generic structure of engines
At lowest level that is not true very few things can do on a screen other than
change a pixel
Various models slot in at different levels

M. Taylor: distinction between what we do to develop the the science, improve
our understanding of how viz works; and what we do to improve an interface now,
these are not opposing goals

V. Taylor: guidelines not absolutes
Varga: results against guidelines
V. Taylor: have constrained set of things you can handle

More talk about getting users interested in new features
Varga: will need more training

Wright: not happy with sonar , needs more iterations
Jacobson her problem is getting the iteration started

V. Taylor: considering the points of the last few days, what do people think of the
IST-05 model?

Smestad says yes good general framework, not inconsistent with guidelines
Jacobson: works differently at different levels of hierarchy
people can be part of model
V. Taylor: how do we get out of this organization at the next levels
What happens if we have a peer relationship

M. Taylor: each one appears in the dataspace of the other

V. Taylor: something about hierarchy

M. Taylor: people who are affecting what you are trying to do are engines, whether
hierarchical or not, engines can be people or silicon

V. Taylor: one of problems with creating chapter on engines, was this refinement
on hierarchy

Smestad: not surprising that the model works; based on cybernetic principles initiated
19th century and refined by Norbert Wiener

V. Taylor: chief sonar operators, his model applied to sonar operator, his engines
include what he views and hears, gets concept, his action is to alert someone
that something is taking place
Understanding and acting fo sonar operator is output device for next level

Hollands scientific model is bad; as a practical model is good

M. Taylor: Six questions evaluator should ask
Iterative nature; use it recursively