About Benchmark

The benchmark used to measure data presented here is “thread metric” benchmark. This benchmarks is designed to test performance of kernel facilities that are regularly used by the application software and their performance will thus directly affect speed of execution of the application.

Our interest into this test suite was based primary on the fact that it benchmarks features which are inherently living inside the kernel and trigger thread switching. Another factor is that the results of these benchmarks can (under ideal conditions) be compared across various systems.

Due to the fact that memory allocation is not part of CMRX kernel and shall be handled by userspace, we removed memory allocator test from the benchmark. Two interrupt-related tests were also removed. What remains are following 5 tests:

These eight benchmarks are:

  1. Basic No-op test. This test does nothing, just increments a timer. It sserves as a calibration. All systems executing this test should produce roughly the same result.
  2. Message processing test. This test sends message via queue, measuring how fast queue subsystem of RTOS is.
  3. Synchronization test. In this test, synchronization subsystem is stressed via semaphore blocking and unblocking.
  4. Cooperative scheduling test. In this test threads are yielding CPU voluntarily. Speed and fairness of scheduling is tested.
  5. Preemptive scheduling test. Similar to above, yet threads yield CPU based on their priority. Similarly as above, speed and fairness is tested.

Its worth mentioning that this benchmark is not made to test contention. In almost all benchmarks, subsystems are stressed using single thread. So when semaphores or mutexes are tested, all accesses come from single thread. The aim of these benchmarks is to test sheer overhead of kernel services in happy path scenarios.

Scoring

Each benchmark test is configured to run for 30 seconds. Amount of repetitions of testing loop is measured. Thus the higher the score, the better.

Testing Hardware

Presented benchmark results were measured using STM32L432KBU6 microcontroller with code running from flash memory and CPU clocked to run at 80MHz.

Toolchain

All tests are being built on AMD64 machine running Linux. Toolchain used to build all tests is: arm-none-eabi-gcc (Arm GNU Toolchain 14.2.Rel1 (Build arm-14.52)) 14.2.1 20241119.

← Performance History CMRX →