Integration and Certification Tests

Integration tests use a server and a controller or controllers to test the behavior of a device. Certification tests are all integration tests. For certified products, the device under test (DUT) is tested against one of the SDK controller implementations (either chip-tool or the python-based controller, depending on the test type). For software component certification, the software component is tested against a sample device built from the SDK.

Certification tests require an accompanying certification test plan in order to be used in the certification testing process. More information about test plans can be found in the test plans repository. Integration testing can also be used outside of the certification testing program to test device behavior in the SDK. Certification tests are all run in the CI.

There are two main integration test types:

• YAML

• Python framework

YAML is a human-readable serialization language that uses structured tags to define test steps. Tests are defined in YAML, and parsed and run through a runner that is backed by the chip-tool controller.

The Python framework tests are written in python and use the Mobly test framework to execute tests.

Which test framework to use

Both types of tests can be run through the Test Harness for certification testing, locally for the purposes of development and in the CI for the SDK. The appropriate test framework to use is whatever lets you automate your tests in a way that is understandable, readable, and has the features you need

• YAML

  • pros: more readable, simpler to write, easy for ATLs to parse and understand

  • cons: conditionals are harder (not all supported), no branch control, schema not well documented

• python

  • pros: full programming language, full control API with support for core (certs, commissioning, etc), less plumbing if you need to add features, can use python libraries

  • cons: more complex, can be harder to read