Unit testing#

Why?#

  • MUCH faster than integration tests.

  • Runs as part of build process.

  • Allows testing specific error conditions that are difficult to trigger under normal operating conditions.

    • e.g. out of memory errors etc.

  • Allows testing different device compositions without defining multiple example applications.

    • e.g. feature combinations not in example apps.

Unit testing in the SDK using pw_unit_test#

This SDK uses Pigweed unit test (pw_unit_test), which is an implementation of GoogleTest. For more information see the pw_unit_test documentation or the GoogleTest documentation.

Simple unit tests#

The following example demonstrates how to use pw_unit_test to write a simple unit test. Each test function is defined using TEST(NameOfFunction). The set of test functions in a given source file is called a “suite”.

#include <pw_unit_test/framework.h>

TEST(YourTestFunction1)
{
    // Do some test things here, then check the results using EXPECT_*
    SomeTypeX foo;
    foo.DoSomething();
    EXPECT_EQ(foo.GetResultCount(), 7);
    foo.DoSomethingElse();
    EXPECT_EQ(foo.GetResultCount(), 5);

    // If you want to abort the rest of the test upon failure, use ASSERT_*
    SomeTypeY * ptr = foo.GetSomePointer();
    ASSERT_NE(ptr, nullptr);
    ptr->DoTheThing();  // Won't reach here if the ASSERT failed.
}

TEST(YourTestFunction2)
{
    // Do some test things here, then check the results using EXPECT_*
    SomeTypeZ foo;
    foo.DoSomething();
    EXPECT_EQ(foo.GetResultCount(), 3);
}

See TestSpan.cpp for an example of a simple unit test.

In the above example there are no fixtures or setup/teardown behavior.

Test fixtures and setup/teardown behavior#

If your tests need fixtures or some kind of setup/teardown you will need to define a test context that derives from ::testing::Test. Each of your test functions will be defined with TEST_F(NameOfTestContext, NameOfFunction). The following example demonstrates how to use pw_unit_test to write a unit test that uses fixtures and setup/teardown behavior.

#include <pw_unit_test/framework.h>

class YourTestContext : public ::testing::Test
{
public:
    // Performs shared setup for all tests in the test suite.  Run once for the whole suite.
    static void SetUpTestSuite()
    {
        // Your per-suite setup goes here:
        sPerSuiteFixture.Init();
        ASSERT_TRUE(sPerSuiteFixture.WorkingGreat());
    }

    // Performs shared teardown for all tests in the test suite.  Run once for the whole suite.
    static void TearDownTestSuite()
    {
        // Your per-suite teardown goes here:
        sPerSuiteFixture.Shutdown();
    }

protected:
    // Performs setup for each test in the suite.  Run once for each test function.
    void SetUp()
    {
        // Your per-test setup goes here:
        mPerTestFixture.Init();
        ASSERT_TRUE(mPerTestFixture.WorkingGreat());
    }

    // Performs teardown for each test in the suite.  Run once for each test function.
    void TearDown()
    {
        // Your per-test teardown goes here:
        mPerTestFixture.Shutdown();
    }

private:
    // Your per-suite and per-test fixtures are declared here:
    static SomeTypeA sPerSuiteFixture;
    SomeTypeB mPerTestFixture;
};
// Your per-suite fixtures are defined here:
SomeTypeA YourTestContext::sPerSuiteFixture;

TEST_F(YourTestContext, YourTestFunction1)
{
    // Do some test things here, then check the results using EXPECT_*
    mPerTestFixture.DoSomething();
    EXPECT_EQ(mPerTestFixture.GetResultCount(), 7);
    sPerSuiteFixture.DoSomething();
    EXPECT_EQ(sPerSuiteFixture.GetResultCount(), 5);

    // If you want to abort the rest of the test upon failure, use ASSERT_*
    SomeTypeC * ptr = mPerTestFixture.GetSomePointer();
    ASSERT_NE(ptr, nullptr);
    ptr->DoTheThing();  // Won't reach here if the ASSERT failed.
}

TEST_F(YourTestContext, YourTestFunction2)
{
    // Do some test things here, then check the results using EXPECT_*
    mPerTestFixture.DoSomethingElse();
    EXPECT_EQ(mPerTestFixture.GetResultCount(), 9);
}

A loopback messaging context for convenience#

If you need messaging, there is a convenience class Test::AppContext that you can derive your test context from. It provides a network layer and a system layer and two secure sessions connected with each other. The following example demonstrates this.

#include <app/tests/AppTestContext.h>

class YourTestContext : public Test::AppContext
{
public:
    // Performs shared setup for all tests in the test suite.  Run once for the whole suite.
    static void SetUpTestSuite()
    {
        AppContext::SetUpTestSuite();  // Call parent.
        VerifyOrReturn(!HasFailure());  // Stop if parent had a failure.

        // Your per-suite setup goes here:
        sPerSuiteFixture.Init();
        ASSERT_TRUE(sPerSuiteFixture.WorkingGreat());
    }

    // Performs shared teardown for all tests in the test suite.  Run once for the whole suite.
    static void TearDownTestSuite()
    {
        // Your per-suite teardown goes here:
        sPerSuiteFixture.Shutdown();

        AppContext::TearDownTestSuite();  // Call parent.
    }

protected:
    // Performs setup for each test in the suite.  Run once for each test function.
    void SetUp()
    {
        AppContext::SetUp();  // Call parent.
        VerifyOrReturn(!HasFailure());  // Stop if parent had a failure.

        // Your per-test setup goes here:
        mPerTestFixture.Init();
        ASSERT_TRUE(mPerTestFixture.WorkingGreat());
    }

    // Performs teardown for each test in the suite.  Run once for each test function.
    void TearDown()
    {
        // Your per-test teardown goes here:
        mPerTestFixture.Shutdown();

        chip::app::EventManagement::DestroyEventManagement();
        AppContext::TearDown();  // Call parent.
    }

private:
    // Your per-suite and per-test fixtures are declared here:
    static SomeTypeA sPerSuiteFixture;
    SomeTypeB mPerTestFixture;
};
// Your per-suite fixtures are defined here:
SomeTypeA YourTestContext::sPerSuiteFixture;

TEST_F(YourTestContext, YourTestFunction1)
{
    // Do some test things here, then check the results using EXPECT_*
}

TEST_F(YourTestContext, YourTestFunction2)
{
    // Do some test things here, then check the results using EXPECT_*
}

You don’t have to override all 4 functions SetUpTestsuite, TearDownTestSuite, SetUp, TearDown. If you don’t need any custom behavior in one of those functions just omit it.

If you override one of the setup/teardown functions make sure to invoke the parent’s version of the function as well. AppContext::SetUpTestSuite and AppContext::SetUp may generate fatal failures, so after you call these from your overriding function make sure to check HasFailure() and return if the parent function failed.

If you don’t override any of the setup/teardown functions, you can simply make a type alias: using YourTestContext = Test::AppContext; instead of defining your own text context class.

Best practices#

  • Try to use as specific an assertion as possible. For example use these

    EXPECT_EQ(result, 3);
EXPECT_GT(result, 1);
EXPECT_STREQ(myString, "hello");

    instead of these

    EXPECT_TRUE(result == 3);
EXPECT_TRUE(result > 1);
EXPECT_EQ(strcmp(myString, "hello"), 0);

  • If you want a test to abort when an assertion fails, use ASSERT_* instead of EXPECT_*. This will cause the current function to return.

  • If a test calls a subroutine which exits due to an ASSERT failing, execution of the test will continue after the subroutine call. This is because ASSERT causes the subroutine to return (as opposed to throwing an exception). If you want to prevent the test from continuing, check the value of HasFailure() and stop execution if true. Example:

    void Subroutine()
{
    ASSERT_EQ(1, 2);  // Fatal failure.
}

TEST(YourTestContext, YourTestFunction1)
{
    Subroutine();  // A fatal failure happens in this subroutine...
    // ... however execution still continues.
    print("This gets executed");
    VerifyOrReturn(!HasFailure());
    print("This does not get executed");
}

  • If you want to force a fatal failure use FAIL(), which will record a fatal failure and exit the current function. This is similar to using ASSERT_TRUE(false). If you want to force a non-fatal failure use ADD_FAILURE(), which will record a non-fatal failure and continue executing the current function. This is similar to using EXPECT_TRUE(false).

  • ASSERT_* and FAIL will only work in functions with void return type, since they generate a return; statement. If you must use these in a non-void function, instead use EXPECT_* or ADD_FAILURE and then check HasFailure() afterward and return if needed.

  • If your test requires access to private/protected members of the underlying class you’re testing, you’ll need to create an accessor class that performs these operations and is friended to the underlying class. Please name the class chip::Test::SomethingTestAccess where Something is the name of the underlying class whose private/protected members you’re trying to access. Then add friend class chip::Test::SomethingTestAccess; to the underlying class. Make sure your test’s BUILD.gn file contains sources = [ "SomethingTestAccess.h" ]. Before creating a new TestAccess class, check if one already exists. If it does exist but doesn’t expose the member you need, you can add a function to that class to do so. Note that you should make these functions as minimal as possible, with no logic besides just exposing the private/protected member.

Compiling and running#

  • Add to src/some_directory/tests/BUILD.gn

    • Example

      chip_test_suite("tests") {
    output_name = "libSomethingTests"

    test_sources = [
        "TestSuite1.cpp",
        "TestSuite2.cpp",
        // Other test source files go here.
    ]

    sources = [
        // Non-test source files go here.
    ]

    cflags = [ "-Wconversion" ]

    public_deps = [
        // Dependencies go here.
    ]
}

    • Another example: src/lib/support/tests/BUILD.gn

  • Build and run all tests with ./gn_build.sh

    • CI runs this, so any unit tests that get added will automatically be added to the CI.

  • Test binaries are compiled into:

    • out/debug/<host_compiler>/tests

    • e.g. out/debug/linux_x64_clang/tests

  • Tests are run when ./gn_build.sh runs, but you can run them individually in a debugger from their location.

Debugging unit tests#

  • After running ./gn_build.sh, test binaries are compiled into

    • out/debug/<host_compiler>/tests

    • e.g. out/debug/linux_x64_clang/tests

  • Individual binaries can be run through regular tools:

    • gdb

    • valgrind

    • Your favorite tool that you tell everyone about.

Utilities#

We have a small number of unit testing utilities that should be used in unit tests.

Consider adding more utilities for general use if you require them for your tests.

Mock clock#

The mock clock is located in src/system/SystemClock.h as System::Clock::Internal::MockClock.

To use the mock clock, use the chip::System::SystemClock() function as normal. In the test, instantiate a MockClock and use the SetSystemClockForTesting to inject the clock. The Set and Advance functions in the MockClock can then be used to set exact times for testing. This allows testing specific edge conditions in tests, helps reduce test flakiness due to race conditions, and reduces the time required for testing as tests no long require real-time waits.

TestPersistentStorageDelegate#

The TestPersistentStorageDelegate is an in-memory version of storage that easily allows removal of keys, presence checks, etc. It is available at src/lib/support/TestPersistentStorageDelegate.h