The taskgraph is built from a YAML file. This file has two top-level properties: components
and tasks
. The full list of tasks is defined by the tasks
object; each task is an object with a single property representing the task with the corresponding value an object representing the task properties. Each task requires the following top-level properties:
provisionerId
: String. Name of Taskcluster provisionerschedulerId
: String. Name of Taskcluster schedulerdeadline
: String. Time until the task expiresimage
: String. Name of docker image to use for taskmaxRunTime
: Number. Maximum time in seconds for which the task can run.artifacts
: Object. List of artifacts and directories to upload; see Taskcluster documentation.command
: String. Command to run. This is automatically wrapped in a run_tc commandoptions
: Optional Object. Options to pass into run_tctrigger
: Object. Conditions on which to consider task. One or more of following properties:schedule-if
: Optional Object. Conditions on which task should be scheduled given it meets the trigger conditions.run-job
: List. Job names for which this task should be considered, matching the output from ./wpt test-jobs
env
: Optional Object. Environment variables to set when running task.depends-on
: Optional list. List of task names that must be complete before the current task is scheduled.description
: String. Task description.name
: Optional String. Name to use for the task overriding the property name. This is useful in combination with substitutions described below.download-artifacts
: Optional Object. An artifact to download from a task that this task depends on. This has the following properties:task
- Name of the task producing the artifactglob
- A glob pattern for the filename of the artifactdest
- A directory reltive to the home directory in which to place the artifactextract
- Optional. A boolean indicating whether an archive artifact should be extracted in-place.Using the above syntax it‘s possble to describe each task directly. But typically in a taskgraph there are many common properties between tasks so it’s tedious and error prone to repeat information that's common to multiple tasks. Therefore the taskgraph format provides several mechanisms to reuse partial task definitions across multiple tasks.
The other top-level property in the taskgraph format is components
. The value of this property is an object containing named partial task definitions. Each task definition may contain a property called use
which is a list of components to use as the basis for the task definition. The components list is evaluated in order. If a property is not previously defined in the output it is added to the output. If it was previously defined, the value is updated according to the type:
For example
components: example-1: list_prop: - first - second object_prop: key1: value1 key2: base_value example-2: list_prop: - third - fourth object_prop: key3: - value3-1 tasks: - example-task: use: - example-1 - example-2 object_prop: key2: value2 key3: - value3-2
will evaluate to the following task:
example-task: list_prop: - first - second - third - fourth object_prop: key1: value1 key2: value2 key3: - value3-1 - value3-2
Note that components cannot currently define use
properties of their own.
Components and tasks can define a property vars
that holds variables which are later substituted into the task definition using the syntax ${vars.property-name}
. For example:
components: generic-component: prop: ${vars.value} tasks: - first: use: - generic-component vars: value: value1 - second: use: - generic-component vars: value: value2
Results in the following tasks:
first: prop: value1 second: prop: value2
Instead of defining a task directly, an item in the tasks property may be an object with a single property $map
. This object itself has two child properties; for
and do
. The value of for
is a list of objects, and the value of do
is either an object or a list of objects. For each object in the for
property, a set of tasks is created by taking a copy of that object for each task in the do
property, updating the object with the properties from the corresponding do
object, using the same rules as for components above, and then processing as for a normal task. $map
rules can also be nested.
Note: Although $map
shares a name with the $map
used in json-e (used. in .taskcluster.yml
), the semantics are different.
For example
components: {} tasks: $map: for: - vars: example: value1 - vars: example: value2 do: example-${vars.example} prop: ${vars.example}
Results in the tasks
example-value1: prop: value1 example-value2: prop: value2
Note that in combination with $map
, variable substitutions are applied twice; once after the $map
is evaluated and once after the use
statements are evaluated.
A common requirements for tasks is that they are “chunked” into N partial tasks. This is handled specially in the syntax. A top level property chunks
can be used to define the number of individual chunks to create for a specific task. Each chunked task is created with a chunks
property set to an object containing an id
property containing the one-based index of the chunk an a total
property containing the total number of chunks. These can be substituted into the task definition using the same syntax as for vars
above e.g. ${chunks.id}
. Note that because task names must be unique, it's common to specify a name
property on the task that will override the property name e.g.
components: {} tasks: - chunked-task: chunks:2 command: "task-run --chunk=${chunks.id} --totalChunks=${chunks.total}" name: task-chunk-${chunks.id}
creates tasks:
task-chunk-1: command: "task-run --chunk=1 --totalChunks=2" task-chunk-2: command: "task-run --chunk=2 --totalChunks=2"
The overall processing model for tasks is as follows:
At each point after maps are evaluated tasks must have a unique name.