Inter-Process Communication

What is Inter-Process Communication?

Inter-Process Communication (IPC) is how separate programs talk to each other. Since processes in CMRX are isolated and can’t directly access each other’s memory, they need special ways to share information and work together.

CMRX does not follow the typical approach of kernel providing several communication/synchronization primitives, like queues and events. Instead it provides two fundamental IPC mechanisms:

• Remote Procedure Calling - to call a procedure from another process
• Notification - to synchronize two or more processes

IPC mechanisms present in traditional RTOSes are then implemented as userspace services based on IPC mechanisms provided by the kernel itself, such as queue server.

Types of Communication in CMRX

1. Remote Procedure Calls (RPC)

Remote procedure call allows one process to call a function from another process. Normally, this is not possible: Another process works with its own memory and this memory is not accessible from current process. If you call that function directly, you’ll get a memory management error. If you call this function via RPC mechanism, the kernel will temporarily give your thread permissions to access callee process’ memory. These permissions are revoked when the function you called returns.

How it works:

From the user’s point of view, using remote procedure calls works like direct function call. You can pass arguments to the function you call via RPC and this function can return value. There are two major differences:

• With RPC, the caller references service object and one of its methods rather directly using function’s address
• RPC is performed via system call rather than directly calling the function

WHy such design?

The service object approach is similar to C-with-objects approach where structures contain function pointers. Main advantage of this approach is that the coupling between calling and called code is broken and contract is established between them. This is very cheap abstraction layer as all the overhead is one added level of indirection.

It gives service implementor the freedom to provide whatever implementation they want or need, as long as the contract is not broken. Any caller code that is consuming this contract won’t be affected by any such change. Thus, coupling is broken.

System call is used for pragmatic reasons - to check if caller has right to do the call and to reconfigure memory protection hardware to give the calling thread access to the callee memory.

That said, despite this mechanism is called RPC, there is no marshalling of data anywhere in the process. Thus all the overhead boils down to performing system calls. Which your software performs hundreds to thousands every second anyway!

In code:

How the server looks like:

// Temperature sensor interface definition
typedef struct {
    void (*configure)(INSTANCE(this), long interval, short scale);
    int (*get)(INSTANCE(this));
} TemperatureAPI;

// Temperature service object definition
typedef struct {
    const TemperatureAPI * vtable;
    long interval;
    /* ... */
} TemperatureSensor;

VTABLE TemperatureAPI temperature_api = { /* provide implementations of API */ };
/* Service instance */
TemperatureSensor temperature_sensor = { .vtable = &temperature_api, /* initialization */ };

How the client looks like:

// In Control Process
int temp = rpc_call(temperature_sensor, get); // Calls get method in temperature sensor object
if (temp > 250) {
    rpc_call(fan, set_power, true); // Calls function in fan object
}

As you can see, all the RPC code is just plain C code. No interface definition language or external parser is involved. You don’t have to parse your source code via external tool or auto-generate bindings. The whole mechanism is also type-checked at compile time, so if you mismatch type or count of function arguments, you’ll get compiler error the same way as if you mismatched arguments of a function call.

Benefits:

• Loose coupling between caller and callee - target process not explicitly mentioned
• Type-safe (compiler checks that you’re performing the call correctly similarly to how function calls are performed)
• Synchronous (calling process waits for response)
• Secure (kernel checks that the object and method you are calling are valid RPC callables)
• Safe (only callee method will ever get access to callee process memory, caller can never access it directly)

2. Shared Memory

Shared memory is a piece of memory that can be used to pass data between two programs without the need of kernel involvement or without copying this data between processes. Unlike normal RTOSes, shared memory in CMRX is not persistent. Memory gets shared only during RPC call execution.

How it works:

SHARED uint8_t buffer[64];

void control_loop() {
    /* ... */
    rpc_call(network, send, buffer); // RPC method has access to buffer passed via pointer
}

When send method of network service is called, it gets access to the buffer. This way network service can read data directly from caller’s memory space eliminating the need to copy the data. Network service is only granted access to data explicitly marked as SHARED and nothing else which greatly reduces risk of damaging caller process memory. Sharing is only active while the call is in progress, which conserves scarce resources of the CPU.

When to use:

• Large amounts of data (like images or audio)
• Data are persistent and not specific to particular call (in this case buffer can be allocated on stack)

Safety features:

• Access control (memory is accessible only during RPC call and only by the thread which processes the call)
• Memory protection (can’t access areas not explicitly shared)