Skip to content

Message Broker for Linux & Windows

Part of the Streaming Integration Platform Base Environment, the Message Broker application runs on the same system as your DBL application. It receives the CDC messages generated by I/O hooks that are added to your application and is responsible for reliably forwarding those messages to the Kafka CDC topic. It is CRITICAL that the Message Broker and Kafka are available at all times when there is the possibility of any activity in your DBL application that could result in changes to replicated data.

The same Message Broker applciation is used on Linux and Windows, where it usually started as a service during system boot. There is a different Message Broker for OpenVMS systems.

Profile Name

When the Message Broker starts it determines the name of the configuration profile to use, which in turn is used to locate the configuration files to be used. The following mechanisms, listed in order of highest to lowest precedence, can be used to define the name of the profile to use:

  • Command line options --config-profile <name> or -p <name>.
  • Environment variable SIP_CONFIG_PROFILE=<name>.
  • (Linux only) The value of PROFILE=<name> read from /etc/sysconfig/sip.conf
  • The default profile name default.

Configuration Directory

The configuration files required by the Message Broker are located in a configuration directory, the location of which can be determined through several mechanisms, listed in order of highest to lowest precedence:

  1. Command line options --config-dir <path> or -d <path>.
  2. Environment variable SIP_CONFIG_PATH=<path>.
  3. (Windows only) C:\ProgramData\Synergex\SIP
  4. Linux default: /etc/synergex/sip

The profile name is appended as a sub directory to the configuration directory. For example, if the configuration profile being used is default and the configuration directory is /etc/synergex/sip, the configuration files should be placed in /etc/synergex/sip/default.

Configuration Files

The Message Broker requires two configuration files:

  • MasterConfig.json
  • iceoryx2.toml

The MasterConfig.json file can contain configuration information for various Streaming Integration Platform components and its actual content will depend on which components are being used in any given environment. The Message Broker utilizes information from the Kafka sections of MasterConfig.json which describes the location and configuration of the Kafka-compatible server(s) and topics being used.

Configuration Server

Because, depending on your environment and use case, the same configuration files can be required by several different tools that may be operating on several different servers, and even on different operating systems, the Streaming Integration Platform provides the Configuration Server, which can be used, amongst other things, to centralize the maintenance of these configuration files.

If the Configuration Server is in use, the Message Broker needs to be able to locate and interact with it. For that to happen it needs two pieces of information:

  • Profile name (already described above)
  • Configuration Server URL

The Configuration Server URL can be specified in obe of the following ways, listed in order of highest to lowest precedence:

  • Command line option --config-server-url <url>.
  • Environment variable SIP_CONFIG_SERVER <url>.

If the Configuration Server URL is not specified then the configuration files must be manually provided and maintained in the appropriate configuration directory.

When correctly configured, the Message Broker will, during startup, connect to and authenticate with the Configuration Server and download copies of both configuration files, which will then be stored in the local configuration directory, overwriting any files that may currently be present. This ensures that the latest version of the configuration files are available each time the Message Broker starts.

If for any reason the Message Broker is unable to establish a connection to the Configuration Server, it will fall back to using the configuration files that are already present in the local configuration directory.

Note that the TLS certificate used by the Configuration Server to provide secure HTTPS communications must be trusted by the system where your DBL application and the Message Broker are running.

Other Command Line Options

The Message Broker has the following additional command line options:

Additional options
  --verbose  or  -v
      Enable verbose logging.

  --help  or  -h
      Display command line options and other help information.

Example Message Broker Startup

Example Message Broker startup
===============================================================================
Streaming Integration Platform: Message Broker
===============================================================================

Config profile          : development
MasterConfig source     : ConfigServer
Config Server URL       : https://localhost:5120
Kafka broker(s)         : 192.168.200.2:9092
CDC topic               : DEFAULT_REPLICATOR
Snapshot request topic  : DEFAULT_REQUEST_SNAPSHOT
Snapshot response topic : DEFAULT_SNAPSHOT
S3 endpoint             : https://192.168.200.2:9000
S3 access_key           : sipagent
S3 secret_key           : ********
S3 region               : us-east-1
S3 bucket               : development

Bootstrap probe OK: 192.168.200.2:9092 -> 192.168.200.2:9092
No global ID bootstrap value found during producer creation:
creating BaseConsumer for GlobalIdAllocator bootstrap defaulting to 0

Ready for messages