Message Broker

The Message Broker is a local process running on the origin system, and it is critical that it is always operating and available whenever any part of the origin application is active.

The Message Broker's job is to receive event messages from the I/O hooks and forward them to the Kafka main event topic. But as the Kafka environment is provided by remote network servers, the Message Broker is also responsible for caching change messages locally in the event that Kafka is unavailable, monitoring the environment to determine when Kafka availability is restored, and then replaying those cached messages to Kafka before resuming real-time operations.

OpenVMS Cluster Environments

If the origin application is hosted by an OpenVMS cluster environment consisting of multiple active nodes, an instance of the Message Broker will run on each of the cluster nodes. However, in order to maintain perfect ordering, only one of those processes will ever be active and responsible for forwarding messages to Kafka. The non-active processes will monitor the active process; if it fails, one of the other processes will become active.

The Message Broker for Windows and Linux is written in Rust. The Message Broker for OpenVMS is written in VMS-C.