The XML Schema TS consists of tests contributed by vendors and other
developers in the
community. The TS is coordinated and supervised by the
XML Schema WG (WG) and NIST.

The TS is intended to be used as a tool to aid implementers in
developing software that supports W3C XML Schema. Validation and
certification of such software are outside the scope of the TS. The
tests and test suite are provided for information and assistance
only. However, the intent is to publish as comprehensive, functional
and
general a test suite as possible.

For more information on the current status of the TS, please visit
the XML
Schema TS home page,
which contains additional information and
pointers to relevant resources.

Structure of the Test Suite

The TS is subdivided into test collections, one for each version of the
Recommendation for which tests are available (thus, if tests were
available for two versions of the Recommendation, the TS would contain
two test collections).

Each test collection consists of one or more sets of tests, which may
be grouped, for example, according to the originator of the test set,
the subject area of the test set, and/or the date upon which the test
set was incorporated into the TS.

Finally, the tests in each set consist of at least one schema
document, accompanied by zero or more instance documents to be
validated against it.

The basis for evaluation of a processor under test is the comparison
between the validation outcome it returns for each of the individual
schema and instance documents in the TS and the expected validation
outcome for that document, as expressed in the TS
documentation/metadata.

Documentation/Metadata

The TS is described, and its constituent tests defined, by a (set of)
metadata file(s). The metadata file is an XML document which conforms
to the TS schema. Thus the TS as a
whole is documented by the aggregation of the documentation submitted
with each contribution of tests.

The functions of the metadata file include:

Definition of each test: the one or more schema and zero or more
instance documents that constitute the test, plus a unique identifier
for the test

Expected validation outcome of each test document

Status and status history of each test document within the TS
(accepted, stable or disputed - see the TS
Process Document for details)

Traceability of tests (via links to the relevant section(s) of
the Recommendation/errata)

Using the Test Suite

Check the README file at the top level of the TS directory
structure for the most uptodate information on the release

Create a test harness to navigate the metadata document,
directing the processor under evaluation to each desired test
(alternatively, direct the processor under evaluation via a browser
using the browsable test report)

(Optional but preferred) produce a test result report to be
forwarded to the TS Task Force (see below)

NOTE:
The schema for test submissions and result reporting is intended
to be sufficiently documented to enable both the construction of a test
harness and the compilation of a result report with the minimum effort.

Reporting test results

The Working Group strongly encourages all members of the community
to report the results obtained when exercising processors against the
TS
or any part thereof. Such feedback gives the WG valuable insight into
such issues as the relevance to the community of various parts of the
Recommendation, which parts are open to conflicting interpretation, and
so on.

The WG reserves the right to publish reported results. Access to
results is determined by the vendor or developer responsible for the
creation of the processor for which results are being reported. Two
levels of access are offered:

W3C Members. Information
on the origin of the test results will be available only to members of
the WorldWide Web Consortium. Outside the W3C, the results may be
published only in such a way that the identity of the processor which
produced the results, and of the vendor or developer which produced the
processor, are not divulged, e.g. as part of aggregated statistical
information.

Public. The reported
results will be publicly available at the TS
website.

The level of access permitted to the results is stipulated by the submitter in the result report.

Note: If any developer or vendor
feels that this level of access to results does not offer sufficient
confidentiality to permit them to report results for their own
processor, they are encouraged to contact the TS Task Force through the
TS website.

The tests and the
accompanying documentation must be well-formed XML; the documentation
must conform to the TS schema,
which outlines such things as the part of
Recommendation under test, functionality of test, location of test, and
so on.

The mechanism for test
submission is as follows:

The test, or series of tests, is submitted to the framework. The
submitter should also send notification to the XML Schema TS mailing
list to indicate that they have submitted a contribution to the TS
framework. Submitting parties are also encouraged to check the tests
they are submitting against the "wishlist" of desired tests.

Receipt of the contribution is acknowledged by the TS Task Force.
This is done via email directly to the submitter, with a copy to
the XML Schema TS mailing list.