This Wiki page is edited by participants of the HTML Accessibility Task Force. It does not necessarily represent consensus and it may have incorrect information or information that is not supported by other Task Force participants, WAI, or W3C. It may also have some very useful information.

Statement of Purpose

This document is an attempt to gather concerns (general and specific)
from the public-html-a11y list that may be relevant to any usability evaluation that we possibly
may undertake to investigate and resolve Table Summary HTML Issue 32.

General Usability Evaluation Questions Drawn From Discussion

Matt

How can study data be fairly analyzed by humans who are frustratingly
analog?

John

Can findings be based on publicly available data?

Ian

Is formal
usability testing of authors followed by double-blind studies of AT users a good evaluation
technique? (Questioning during testing seems to be problematic.)

Is having an independent researcher who is not aware of the testing "grade" the tests a good idea? (Ian "might" be able to get resources to do testing early in 2010.)

Laura

Would it be in-scope for this task force to do surveys of users? (Like the WebAIM surveys geared to the table summary Issue with questions like Roger Johansson asked on his blog.)

Could
the W3C WBS
survey tool be used to conduct surveys of users?

Do we have the resources and does the group want to do full blown
usability testing?

Are there any accessibility task force members willing to collaborate
on doing usability evaluations?

Leif

What is the feasibility of the task force doing a usability test?
Do we have the needed resources?

Can we include authors as well as assistive technology users as
participants in any studies?

Can goals be specified so everyone agrees upon them?

How can evaluations be fairly designed, written, conducted, and measured so that
this task force can accept the findings?

Cynthia

Is it possible to include business concerns in a usability study?
Can we include VPs of marketing in the user group? Change the
authoring task to say that every addition pixel costs
$1 million? (true story) Would that be valid?

Usability Testing Definitions

A Concern is a question about usability.

A Goal is a declarative statement resulting from
a question of concern. A goal establishes an objective to measure.

Specific Table Summary Concerns (questions about usability)

Matt

Will a table summary replacement that defaults to visible not be
used by authors because it's presented visually?

What is the best way to reconcile aesthetics and data with one
another?

If people are taught to write good table summary
values will they write bad summary values?

Will people think to add a summary paragraph, if they are they're
not prompted to do so by a tool?

John

What evidence is there that @summary causes harm to
users?

Is meta-data better than no data?

Ian

Would a visible table summary help accessibility?

Does a hidden table summary attribute harm accessibility?

Does a hidden table summary attribute help accessibility?

Roger

Do assistive technology users who use shortcuts to jump straight
to the table miss any information if the table summary information
is provided in paragraphs not explicitly associated with a table?

What is the best way for authors to satisfy stakeholders aesthetics
of not having a table summary information visible while still
making a table accessible?

Leif

What kind of extra table summary information is needed for accessibility?

What is the best way to add that information?

Laura

How can table summary be improved?

Specific Evaluation Goals

To be determined...agreed upon goals of any evaluation would be a
first step (if in fact the group wants to do evaluations). Everything in any evaluation should stem from what the group wants to investigate without preconceptions or bias. Also criterion, methodology, etc could be a huge bone of contention if not agreed upon by the group in advance.

In usability testing, goals are declarative statements resulting from a
question of concern. A goal establishes an objective to measure in an
evaluation.