Test Licensing

If a Contributor is a member of WebApps, there is no need to complete any license forms. However, if a Contributor is not a member of WebApps, the Contributor must grant permission to distribute their contribution under these two licenses, and this requires completing the W3C's license grant form.

Test Repository Structure

The WG uses the W3C's web-platform-tests repository on GitHub for its tests.

Test Case Requirements

If a test is a visual test (not API specific) a Reference Test (aka RefTest) must be created. Note that -ref must be added to the testcase name to identify the reference page. For some information on RefTests, see:

You might get asked about completing W3C's license grant form if your employer is not already a member and you have not done so before. Someone will review your test and you might have to do some changes based on that review.

Test Review and Approval Workflow

The general workflow for test reviews and approval follows the GitHub contribution model and is summarized here:

Pre condition: we assume everyone that is interested in reviewing and/or approving WebApps' tests will set a "Watch" at GitHub so they will be notified when a Pull Request (PR) is made.

To initiate a review, make a Pull Request to the main web-platform-tests/master on GitHub (this will notify all subscribers of the PR). The review initiator can also notify the group of the PR via the public-webapps-testsuite mail list.

Reviewers should review the test code and reply accordingly. If no issues are found, the reviewer should state that so there is a trail a review was done.

After all of the review issues are addressed, the PR will be merged into the repository. The merge may be done by the spec's Test Facilitator or someone that is overseeing the test repository. ^1

The merged tests are considered Approved.

^1 If you are a member of WebApps, and would like to be able to merge Pull Requests after a successful review, ask one of the W3C Staff members to add your GitHub account into the Team group. Good candidates are Robin (darobin), Tobie (tobie) and Mike (MikeSmith).

Testing Roles

Information in this document refers to the following roles:

Contributor / Submitter - someone that contributes one or more test cases

Test Facilitator - the person(s) responsible for maintaining a specification's test suite, including updating test cases, making sure Pull Requests are reviewed, etc. Normally, this role is assumed to the specification's Editor(s). A Test Facilitator will be designated as the Default Assignee for the test suite's bug component in WebApp's Bugzilla database. Test Facilitators are expected to work with Contributors e.g. to resolve quality issues, to eliminate redundant/overlapping asserts, etc.

Testing Coordinator - the person responsible for coordinating all of WebApps' testing efforts. By default, this role is taken by a WG Chair, a WebApps Team Contact or a member of the WG. (In practice, this role may be shared by more than one person.)

Test Suite Status

The easiest way to determine the status of a spec's test suite is to simply ask on the public-webapps-testsuite@w3.org mai list. Additionally, some test suite status information including the spec's Test Facilitator is available in WebApps Publication Status document.

How to Report a Test Case Bug

A bug report for a test case can be reported via any of the following:

File a Bugzilla bug against the appropriate test suite component in WebApps' Bugzilla database. Please identify the test case(s) that has a bug and clearly describe the bug. (If you do not have write access to the bug database, send an e-mail to the list above).