Message Broker

The Message Broker is a local process running on the system of record, and it is critical that it is always operating and available whenever any part of the system of record is active.

The Message Broker's job is to receive event messages from the I/O hooks and forward them to the Kafka CDC topic. But as the Kafka environment is provided by remote network servers, the Message Broker is also responsible for caching change messages locally in the event that Kafka becomes unavailable, monitoring the environment to determine when Kafka availability is restored and then replaying those cached messages to Kafka before resuming real-time operations.

OpenVMS Cluster Environments

If the system of record is an OpenVMS cluster, an instance of the Message Broker will run on each of the cluster nodes. However, to maintain perfect ordering, only one of those processes will ever be active and responsible for forwarding messages to Kafka. The non-active process will monitor the active process; if it fails, one of the other processes will become active.

The Message Broker for Windows and Linux is written in Rust. The Message Broker for OpenVMS is written in VMS C.