Unit testing C code with CMocka

Assert functions

Instead of using the classic assert() call from well-known assert.h, CMocka comes with a small set of practical assert functions. Among them, the ones that I use the most are:

void assert_int_equal(int a, int b);
void assert_string_equal(const char *a, const char *b);
void assert_memory_equal(const void *a, const void *b, size_t size);

Other functions seem useful too, such as assert_null and assert_in_range. You can find the full list of assert variants on the dedicated assert module documentation

Declaring tests

Test declaration is to be done in two steps. First we declare and implement the test function that must have the following type signature:

void test_do_something(void **state) {
    (void) state; /* unused */
}

The void **state pointer is used when you specify setup and teardown functions to the test, which allow to initialize and deconstruct a state for each test. Useful in order to avoid code duplication if you have to do these common steps for multiple independent tests. I do not use it at this moment so I add this little construction in order to avoid GCC to raise warnings about unused variables.

System call mocking with wrap

This allows us to verify that a system call like open() was correctly called, without really calling the system's open() function. So we can mock a lot of system calls like this, typically open(), write(), read() and close() functions. Once the mock is declared we can check that it was called with the good arguments and return success (0) or error (-1) in order to simulate system failures.

This even allows us to check that a function was not called if we did not expect it. If you call a mock without declaring that you expect it to be called, then you will get an error. This is useful if you want to check that you do not perform a read() if the open() function call has failed.

For this system call mocking we will use a GCC linker option named --wrap=.... Since you are more likely to call gcc instead of ld in your compilation process, you can tell gcc to pass the --wrap option to ld by passing the --Wl,--wrap=... argument to GCC.

The --wrap=open notation asks GCC to make every call to the open function to reach the __wrap_open function instead. If for whatever reason you need to use the real open function after this option is passed, then you can call the __real_open function (after declaring its prototype).

Custom function mocking using weak symbol

We can even split our own tests in order to test only one function at a time, hence test decoupling. We can verify that a function correctly calls another function that we already tested before. If we do not perform this tests splitting, then we are not doing unit tests anymore but functional tests, since you will test a function that will call a function that will… you get it.

This splitting really depends on how you want to test your code and on your own taste. Typically, I use wrapping in order to test a function that uses system calls, and then mock this function inside another test unit in order to see that it is correctly called by other functions.

Here --wrap cannot work if the function you want to replace is in the same source file. Indeed, linking to this function is already done, because the compiler has found the function inside the same file. It does not mark it as "unresolved", which is a necessary condition in order to make --wrap ld option work.

We then have to use another trick in order to redefine the function in order to mock it. For this, GCC provides us with a new bypass: weak symbols. By default, when we declare the implementation of a function in a C file, then the function symbol is defined as a strong symbol. Any attempt to redefine this strong symbol will lead to the well know error you may know: the multiple definiton of error.

libhello.a(hello.c.o): In function `open_i2c':
lib/hello.c:15: multiple definition of `open_i2c'
tests_do_something.c.o:tests_do_something.c:12: first defined here

Weak symbols, however, can be redefined. This is exactly what we need in order to test that our high-level function correctly calls our low-level function, while these to functions being implemented in the same source file.

To declare a weak symbol, we can use the GCC __attribute__((weak)) annotation before the function implementation.

We may not be comfortable with the idea that our users can now redefine our internal functions if we export them as weak symbols. A workaround I am using right now is to export functions as weak symbols only during a debug build, which is needed in order to build and run tests in my setup:

#ifdef DEBUG
#define WEAK_FOR_DEBUG __attr__((weak))
#else
#define WEAK_FOR_DEBUG

And we can declare our functions as weak only during debug build like this:

WEAK_FOR_DEBUG
int myfunction(int a, int b)
{
    return a + b;
}

Maybe I should use a TESTING define instead of DEBUG and define it only when building for tests but not for debugging.

Drawbacks: lots of executables

The downside of both of these methods is that once a function is redefined, it will be in the whole test executable. If you want to test a function and the functions it calls then you will need two different executables, and two different test files.

If we correctly test our program, then we are starting to see tens of test executables, with different wrapping and weak symbol redefinitions inside them. Hopefully we can rely on CMake in order to build and launch all of these test executables.

Issues with gcov and system calls mocking

There is a big issue if you mock system calls and activate code coverage: once these compilation flags added, your program will try to write the code coverage report to a binary file when its execution has ended. However in order to write into this file it needs the open() function. If you have already mocked it, then the code coverage tool is going to have big trouble trying to write to its report file.

We can detect this problem by launching the test executable and by looking at its exit code. Sometimes all tests are green but the test program will return 255 (which is -1 mapped onto a uint8_t). This behavior is typical of a write error of code coverage results, that is called once your program has ended. If you observe this then there are good chances that you mocked a system call that gcov needed for its code coverage report.

The solution one proposed me on StackOverflow is to look if our open() mock is called on a file that looks like a code coverage report file. If it is the case then we really open the file in order for gcov to write to it. If the file does not look like a code coverage file then we can mock it for our testing needs.

int __real_open(const char *path, int flags, int mode);
int __wrap_open(const char *path, int flags, int mode)
{
    if (strlen(path) > 5 && !strcmp(path + strlen(path) - 5, ".gcda"))
        return __real_open(path, flags, mode);
    printf("hello from __wrap_open\n");
    return -1;
}

Source code we want to test

Here are the functions we want to test using CMocka.

/* hello.h */

#ifndef HELLO_H
#define HELLO_H

#include <stdint.h>

#define I2C_SLAVE_FORCE 0x0706

int open_i2c(uint8_t i2c_addr);
int do_something();

#endif /* HELLO_H */

And below is the trivial implementation.

/* hello.c */

#include "hello.h"

#include <stdio.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>

#define PATH_BUFFER_SIZE 32


__attribute__((weak))
int open_i2c(uint8_t i2c_bus)
{
    char path[PATH_BUFFER_SIZE];
    int fd;

    snprintf(path, PATH_BUFFER_SIZE, "/dev/i2c-%d", i2c_bus);
    fd = open(path, O_RDWR);

    if (fd < 0)
        return -1;

    return fd;
}

int do_something()
{
    int fd = open_i2c(42);

    if (fd < 0)
        return -1;

    return close(fd);
}

Testing the low-level open_i2c() function

First we want to test that the open_i2c() function we wrote opens the correct file given its i2c-bus argument. For this we create a new test file with the following code:

/* test_open_i2c.c */

#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>

#include <stdio.h>
#include <string.h>

#include "cmocka.h"
#include "hello.h"

/* redefinitons/wrapping */

int __real_open(const char *path, int flags, int mode);
int __wrap_open(const char *path, int flags, int mode)
{
    if (strlen(path) > 5 && !strcmp(path + strlen(path) - 5, ".gcda"))
            return __real_open(path, flags, mode);

    check_expected(path);
    return mock();
}

/* tests */

void test_open_i2c_failure(void **state)
{
    (void) state; /* unused */
    int ret;

    expect_string(__wrap_open, path, "/dev/i2c-99");
    will_return(__wrap_open, -1);

    ret = open_i2c(99);
    assert_int_equal(-1, ret);
}

void test_open_i2c_success(void **state)
{
    (void) state; /* unused */
    int ret;

    expect_string(__wrap_open, path, "/dev/i2c-99");
    will_return(__wrap_open, 42);

    ret = open_i2c(99);
    assert_int_equal(42, ret);
}


const struct CMUnitTest open_i2c_tests[] = {
    cmocka_unit_test(test_open_i2c_failure),
    cmocka_unit_test(test_open_i2c_success),
};


int main(void)
{
    return cmocka_run_group_tests(open_i2c_tests, NULL, NULL);
}

Can you see which one of the two GCC hacks we are going to use in order to redefine the open() system call in these tests? Since this function is undefined in our code we can freely use the -Wl,--wrap=open option! Make your eyes sharp and now read the two tests carefully. See how we tell our open() mock to check its arguments and to return the value we want?

Testing the higher-level do_something() function

For the second test unit we want this time to check that our do_something() function is correctly calling our open_i2c() function and the system's close() function.

/* tests_do_something.c */

#include <stdarg.h>
#include <stddef.h>
#include <setjmp.h>

#include <stdio.h>

#include "cmocka.h"
#include "hello.h"

/* redefinitons/wrapping */

int open_i2c(uint8_t i2c_addr)
{
    check_expected(i2c_addr);
    return mock();
}

int __wrap_close(int fd)
{
    check_expected(fd);
    return mock();
}

/* tests */

void test_do_something_failure(void **state)
{
    (void) state; /* unused */
    int ret;

    expect_value(open_i2c, i2c_addr, 42);
    will_return(open_i2c, -1);

    ret = do_something();
    assert_int_equal(-1, ret);
}

void test_do_something_success(void **state)
{
    (void) state; /* unused */
    int ret;

    expect_value(open_i2c, i2c_addr, 42);
    will_return(open_i2c, 42);

    expect_value(__wrap_close, fd, 42);
    will_return(__wrap_close, 0);

    ret = do_something();
    assert_int_equal(0, ret);
}


const struct CMUnitTest do_something_tests[] = {
    cmocka_unit_test(test_do_something_failure),
    cmocka_unit_test(test_do_something_success),
};


int main(void)
{
    return cmocka_run_group_tests(do_something_tests, NULL, NULL);
}

This time we are redefining our open_i2c() function using the weak symbol redefinition, and still wrapping the external close() system function. See that using this trick we are not calling the real open_i2c() function, and thus we do not have to wrap calls to open() again!

Finally, note that in the failure test case we do not specify that we expect __wrap_close to be called. If you remove the return in the C code and call close if open_i2c has failed, then you will get a CMocka error because you did not tell it what to verify for __wrap_close in this test case.

Bonus: code coverage results

Using the CMake file I presented you, CMake adds a test_coverage target to the generated Makefile. It will clean the code coverage results, launch all the tests using ctest and then generate code coverage results as a set of HTML files.

Yay, this is a 100% code coverage report!

You can see that with these tests we have a full coverage of our code and all branches, since we wrote tests for expected failures too.