Purpose and General Description

Test Case Description Language (TCDL) is an XML-based language defined by BenToWeb
to store metadata for the test cases in the test suites for
Web Content Accessibility Guidelines (WCAG) 2.0.
For each individual test case, it describes:

formal metadata, such as author, date, title and a short description;

technologies (the technologies and technology features used in the test);

test mode and test scenarios for end user evaluation;

the “rules” that are tested (WCAG 2.0 success criteria or other rules);

namespace mappings used in some of the previous sections.

TCDL is defined in
W3CXML Schema
and uses the namespace http://bentoweb.org/refs/TCDL1.1;
the XML Schema is available at
http://bentoweb.org/refs/schemas/tcdl1.1.xsd and contains detailed documentation.
The next section provides a less technical description of TCDL. It also shows which metadata elements TCDL
has in common with those suggested in the
W3C Note on Test Metadata by adding “W3CTM” to the description.

Structure of TCDL

Each test case has a unique identifier (stored as a mandatory ID attribute of the test case description;
W3CTM) that uses the format technology_wcag2_date_x.y_lz_scz_nnn,
where the sections separated by underscores have the following meaning:

technology

the technology for which the test suite is developed: xhtml1, xhtml2, xforms1, etcetera,

wcag2

a constant referring to WCAG 2.0,

date

the date on which the WCAG 2.0 draft was published, in yyyymmdd format,

x.y

the number of WCAG 2.0 guideline, for example 1.1, 1.2, …,

lz

the WCAG 2.0 level: l1 | l2 | l3,

scz

the number of the WCAG 2.0 success criterion, for example sc1, sc2, …,

nnn

the number of the test case (for this success criterion, not the number in the complete test suite), for example 001, 002, …

Formal Metadata

The formal or administrative metadata that can be stored are:

description (mandatory; W3CTM);

title (mandatory; W3CTM);

author (mandatory; W3CTM);

language (mandatory);

rights (mandatory; W3CTM) — usually BenToWeb copyright;

creation date (mandatory);

status (mandatory; W3CTM);

version (optional);

source (optional): if the test file is borrowed from an existing test suite, the test suite and the URL
of the original test file can be documented; in addition, it is possible to indicate

if the borrowed file was modified and

to document these modifications and other comments, and

rights (for example copyrights; W3CTM) related to the borrowed file.

Technologies

BenToWeb uses the term “technology” in the same sense as WCAG 2.0:
a data format, programming or markup language, protocol or API.
TCDL provides the ability to store the following information about technologies used in a test:

the formal specification or recommendation that defines the technology (by name and URL; mandatory);

the technology features (for example, which XHTML elements and/or attributes) that are relevant to the test (optional);

a reference (URL) to a section in the formal specification or a quote from the formal specification
that describes the technology features that are relevant to the test (optional).

Test Case

Purpose

This mandatory section provides a brief explanation of the reason why the test was developed (W3CTM).

Preconditions

The W3C Note on Test Metadata
defines preconditions as conditions that must be met before the test can be successfully executed.
This section was added to TCDL for other users of the language.
It is not used in BenToWeb because test cases map to success criteria instead of tests.
However, the current HTML Test Suite for WCAG 2.0 uses “prerequisite tests”, and these could be described here.

Required Tests

This section describes:

the test mode or modes (mandatory): the ways in which the test case should be evaluated to advance the test case from
“draft” status to “accepted”;
test modes are at least one of the following:

end user,

one accessibility expert,

a group of accessibility experts or

automatic evaluation;

one or more test scenarios for end user evalution (strictly optional, but required if the test mode is “end user”):
a scenario consists of

user guidance (for example, advice on features of a user agent that should be supported by and enabled in a browser),

questions that the user is expected to answer
(yes/no question, open question, Likert scale, multiple choice, or a combination of yes/no and open question),

the experience that the user should have with certain user agents or assistive technologies to evaluate the test, and,

optionally, disabilities to which the test is relevant.

Each scenario has a mandatory identifier that is unique within the test case description.
This identifier is necessary to map end user feedback to the scenarios.

Test Files

This section contains information about each of the test files that are relevant to the rule or success criterion:

Supporting files such as included images, external style sheets and dummy pages to which forms are submitted
(if only the form is relevant to the test) etcetera, are not documented.

Rules

This section contains the “rules” that are tested by a test case.
In BenToWeb, rules are generally WCAG 2.0 success criteria, but it is also possible to reference other sets of “rules”:

WCAG 1.0 checkpoints,

Section 508 provisions,

BITV provisions.

The rules do not provide a direct reference to the relevant online source because it may not always be possible
to reference “rules” directly (for example, because they are not available in HTML).
The rules reference a rule ID in a “Rulesets XML” file (see Rulesets XMLRDDL document),
which in turn references the relevant rule in an online source.

For each rule listed in this section, the following information can be provided:

whether the rule is the “primary” rule of the test case or only listed for informative purposes
(for example, some test cases uses invalid markup in tests that are not about validity, so it is useful to document that checkpoints about validity should not be applied to the test case)
(optional);

the identifier of the rule in “Rulesets XML” (mandatory);

the locations in the test file that are relevant to the rule, with URL, line number, column number
and/or XPath expression (optional);

whether the test case passes or fails the success criterion or rule;

the functional outcome of the test: a description of why the test case passes or fails,
in terms that relate to a user's experience (as opposed to technical comments about source code)
(mandatory);

technical comment on the test: a technical description of why the test case passes or fails,
and other technical information (for example, issues in certain browsers or assistive technologies).

Namespace Mappings

This optional section is important for test cases based on test files that use XML-based technologies.
The technology features (see the section on Technologies above) used in a test case exist in an
XML namespace and
should be referenceable with XPath.
The normal mechanism for mapping namespaces to URIs by declaring them with
xmlns:prefix="namespaceURI" cannot be used for this,
because it is necessary to be able to define empty prefixes and to extract the namespace mappings with a common XMLAPI.

TCDL and W3C Test Metadata

The W3C Working Group Note “Test Metadata” (14 September 2005)
suggests fourteen metadata elements for test suites. The table below shows how these metadata elements map to TCDL elements.
(TCDL elements are identified by means of XPath expressions without a namespace prefix and with the root element — testCaseDescription — omitted,
except for the attributes of that element.)

Mapping of W3C Test Metadata to TCDL

W3C Metadata

TCDL

Comment

Identifier

testCaseDescription/@id

Title

formalMetadata/title

In TCDL, titles are not necessarily unique; they don't duplicate the identifier.

Purpose

testCase/purpose

Description

formalMetadata/description

In TCDL, the description should help the reviewer understand how the test materials are constructed and the behaviour when a user interacts with it.

Status

formalMetadata/status

SpecRef

rules/ruleandtechnology/recommendation

In TCDL, two types of specifications need to be referenced: the accessibility guidelines (“rules”)
and the specifications of the technologies in the test.

Preconditions

testCase/preconditions

Not used in BenToWeb, because test cases map directly to WCAG success criteria instead of tests.

Inputs

testCase/files/file/httpRequest

Only for the special case where it is necessary to (re)create a specific HTTP request.

ExpectedResults

rules/rule/locations/@exptectedResultandrules/rule/functionalOutcome

Version

formalMetadata/version

Not used in BenToWeb; a version control system (CVS) is used instead.

Contributor

-

BenToWeb does not have external contributors, but can borrow test files and document them at
formalMetadata/source.

Rights

formalMetadata/dc:rightsandformalMetadata/source/dc:rights

TCDL can document rights related to a test case and rights related to borrowed materials.

Grouping

-

(Because of the strict filename convention in BenToWeb, some types of grouping can be done by means of filename pattern matching.)

SeeAlso

technology/recommendation(andrules/rule/techComment)

Differences with IBM's TCDL Submission to W3CQA

BenToWeb's TCDL was developed completely independently from the TCDL proposal submitted by IBM to the W3C Quality Assurance Working Group (QAWG) in 2003.
The goal of IBM's TCDL is to catalogue most of the test materials that a W3C Working Group would provide.
It is intended to be used in a test lab “to set up and run the test materials against one or more test subjects” and
“supports automated setup of the system(s) used for testing, automated running of test cases, automated comparison of results, and automated cleanup of the system(s)”.
Like BenToWeb's TCDL, IBM's TCDL is an XML vocabulary,
so metadata can be tranformed in other formats for human or machine consumption.

An IBMTCDL catalogue contains a list a test cases,
whereas a BenToWeb TCDL contains metadata for an individual test case only.
BenToWeb's TCDL is also different because each test case references at least two specifications
(WCAG 2.0 or another ruleset, and a specification of a technology such as XHTML or CSS),
and one of these (WCAG 2.0) cannot be tested fully automatically.

Unfortunately, it is not possible to make a detailed comparison of the two languages because the XML Schema for IBM's TCDL submission is not available
(the link to the Schema in Section 7.4 does not lead to a file).