Kernel Tests

The microkernel design of CMRX RTOS allows for very easy testing of core kernel functionality. The kernel of CMRX is designed not to access the hardware directly. Instead, it assumes presence of the abstract machine it talks to. This design allows easy testing of kernel features due to the lack of direct calls to any specific hardware.

Even in such small kernel as the kernel of CMRX is, the raw hardware management code is a very minor part of the codebase. Yes, we still need to configure hardware registers of memory protection unit, configure timers and swap threads. Yet the bulk of code resides in the implementation of emergent features, such as synchronization, data sharing and prioritization of execution.

What is Kernel?

At first glance, the operating system kernel can seem like a magic piece of code that does some exceptional and almost super-natural work. After all, it allows things like processes and threads, manages memory and manages all the hardware. This must be a very special task, right?

Not at all. It turns out that much of the kernel is mostly just a plain CRUD (Create, Read, Update, Delete) on several data structures. Some of these structures map to hardware which makes their design bound by physical constraints. Other than that there’s nothing really special in the kernel. You have to mind the execution speed of the code and that there are interrupts. Yet most of the time you are just modifying data in RAM.

CRUD can be tested

Thus, as pretty much any other CRUD application, such kernel can also be largely tested. Unlike approach where kernel is randomly touching hardware, testing of CMRX is mostly straightforward. There is a unit test layer, which tests individual function blocks of the kernel API call by API call. Individual test quickly simulate various scenarios which would be tricky or nearly impossible to achieve on real hardware. On the other hand, any scenario we find on real hardware can be set up in testing environment and tested for appropriately.

Below, you can see the unit test coverage of the kernel code as of late 2025. It could certainly be improved. Still, it covers more than with many other systems out there.

The main advantage of this unit test suite is that it is easy to be built and ran on CI/CD infrastructure. There’s no need for attached HiL testing harness, emulators or special libraries. All you need is a host compiler, sources and a couple of lines of code in your automation job. No time spent flashing tests into target hardware, no need to handle stalled USB host controller, confused probe or worry about test hardware FLASH wear and tear.

And so we do. Each and every commit is subject of running unit tests. If we find a bug that is testable with unit tests, we add it as regression test.

As the code under test is a kernel code, we sin a bit. We violate the rule of unit testing that tests the outcome rather than behavior. The reason for this is simple: Many of outcomes are not externally visible. Such visibility either compromises security or is simply not needed for the userspace running CMRX kernel.

This also happens in tests where the code is tested for behavior in scenarios with concurrency involved. Here we have to rely on the knowledge of the internal behavior. On the other hand, it gives us the opportunity to test for correct handling of concurrency. Without real hardware.

Representativeness of off-target testing

Many die-hard embedded developers would say that off-target unit testing (on developer’s machine) is worthless as it doesn’t represent the execution of the code on the target hardware, which may handle the code differently. The statement on different hardware handling same code differently is only partially true. While in generic case, there are C statements which behave different on different machines, most of such behavior is “undefined behavior” by the C standard. The rest can be sorted out by using specific-width types, which arrived with C99, some 26 years ago.

Having code that is free of undefined behavior and uses specific-width types the degrees of interpretation freedom are close to 0. In such environment, if unit test fails on the host machine, the code is objectively faulty and needs fixing. There’s no point in running such code on the target hardware, because it will be faulty there as well.

So, a unit test that fails is a representative sign of a bug in the code. Despite this layer of testing is not able to catch all possible failures, the simplicity of maintenance and speed of execution in CI still make it a worthy tool.

Some of the trickier parts of the CMRX kernel, such as transaction engine were developed exclusively using test-driven development with only tests used to test the library. The transaction library itself turned out to be bug-free on real hardware.

The conclusion is that one of benefits of portable microkernel architecture of the CMRX RTOS kernel is that this kernel is simple to be tested. These tests are cheap to run and maintain and easy to automate. Benefits of automated tests are well-known and CMRX makes them dirty cheap to be ran. This helps to maintain the quality of the code.