Labels

Labels allow you to:

  • Organize tests to be more meaningful to your team and organization.
  • Categorize tests, and therefore, can be used to filter tests within Test Engine.
Screenshot of a test with labels categorizing the test

Labels are created at the test suite level. Therefore, labels belonging to one test suite will not impact the labels associated with other test suites.

Label a test

Labels may be applied to or removed from tests:

Using the Buildkite interface

From the details page of a test (accessible through its test suite's Test page), select Add labels and either:

  • Select a label from the list of existing used in the test's suite.
  • Specify a New label, select its Label color, and select Save.

To remove a label from a test, select Add labels from the test's details page, and from its drop-down, clear the checkbox next to the label.

Using automatic quarantine

Using test states, you can automate the addition and removal of labels when a specific test failure threshold is reached.

Learn more about this feature in Automatic quarantine.

Using execution tags

A test execution tag value can be applied as a label on a test.

Also, when Test Engine detects a change to such a tag's value, its label on the respective test is also updated to this value.

Test Engine only labels tests from execution tag values when their test suite is configured to do so in the suite's Settings > Test labels (tab) page.

Screenshot of configuring suite settings to copy tags to labels

Learn more about test execution tagging in Tags.

A label added to a test through a test execution tag is automatically removed when the tag is removed from the test execution.

Using the REST API

You can label tests using the REST API with the Tests API endpoint. Learn more about this in Add/remove labels from a test.

Filter tests

You can filter tests using labels through the Buildkite interface or REST API.

Using the Buildkite interface

On the test suite's Tests page, either:

  • Enter label:labelname in the search field and select Search.
  • For any existing test with at least one label applied to it, select the test's label > Filter by from its drop down to filter the test suite for all tests with that label applied to them.
Screenshot of filtering by a test label

Using the REST API

You can fetch all tests with a label using the REST API with the Tests API endpoint. Learn more about this in List tests, and by specifying the optional label query string parameter.