testharness.js
test suiteThe test suite for the testharness.js
testing framework.
Install the following dependencies:
Make sure geckodriver
can be found in your PATH
.
Currently, the tests should be run with the latest Firefox Nightly. In order to specify the path to Firefox Nightly, use the following command-line option:
tox -- --binary=/path/to/FirefoxNightly
Alternatively, you may run tools/ci/ci_resources_unittest.sh
, which only depends on Python 2. The script will install other dependencies automatically and start tox
with the correct arguments.
Test cases are expressed as .html
files located within the tests/unit/
and tests/functional/
sub-directories. Each test should include the testharness.js
library with the following markup:
<script src="/resources/testharness.js"></script> <script src="/resources/testharnessreport.js"></script>
This should be followed by one or more <script>
tags that interface with the testharness.js
API in some way. For example:
<script> test(function() { 1 = 1; }, 'This test is expected to fail.'); </script>
The “unit test” type allows for concisely testing the expected behavior of assertion methods. These tests may define any number of sub-tests; the acceptance criteria is simply that all tests executed pass.
Thoroughly testing the behavior of the harness itself requires ensuring a number of considerations which cannot be verified with the “unit testing” strategy. These include:
Functional tests allow for these details to be verified. Every functional test must include a summary of the expected results as a JSON string within a <script>
tag with an id
of "expected"
, e.g.:
<script type="text/json" id="expected"> { "summarized_status": { "message": null, "stack": null, "status_string": "OK" }, "summarized_tests": [ { "message": "ReferenceError: invalid assignment left-hand side", "name": "Sample HTML5 API Tests", "properties": {}, "stack": "(implementation-defined)", "status_string": "FAIL" } ], "type": "complete" } </script>
testharness.js
is expected to function consistently in a number of distinct environments. In order to verify this expectation, each functional test may be executed under a number of distinct conditions. These conditions are applied using WPT's “test variants” pattern. The available variants are defined in the variants.js
file; this file must be included before testharness.js
. Every test must specify at least one variant.