Skip to content

Target Systems

Target systems are systems that receive the replicated data in some form, or that process the stream of CDC messages for some other purpose.

Out of the box the Streaming Integration Platform provides two target environments:

  • Data replication to remote SDMS files.
  • Data replication to Microsoft SQL Server

Data Replication to Remote SDMS Files

The ISAM Agent can be used to achieve the replication of SDMS or RMS ISAM and relative data files to a matching set of data files on one or more remote systems.

The primary use cases for doing so include:

  • Implementing a phased migration to a new platform
  • Off-loading workloads from the primary origin system to other environments
  • Replicating application data to a different system or location for disaster recovery purposes

Data replication to Microsoft SQL Server

The SQL Agent can be used to facilitate the replication of SDMS or RMS ISAM and relative data to a Microsoft SQL Server database.

The primary use cases for doing so include

  • Exposing the data in a form easily consumable by other home-grown or third-party applications, without any significant impact on the origin system
  • Data warehousing or other ETL use cases

Latency

The Streaming Integration Platform is designed to achieve high performance and throughput, resulting in the replication of data in near real time. But regardless of how small they are, there is always some amount of latency between changes to the system-of-record data and the replicated data, and this latency can be impacted by the choice of technologies and deployment environments.

The environment is by definition distributed across multiple systems, usually involving at least three different system environments:

  • The origin system
  • The Kafka (or Redpanda) environment
  • The target system

Ideally the network connections between these environments should be as fast and low latency as possible, but in some cases some or all of these components may be physically separated, and perhaps even cloud based. The latency and throughput of the overall solution will depend heavily on these factors.

Read-Only Access

In both of these use cases cases, the replicated data should generally be considered available for read access only. There is one exception: the data has been replicated to SDMS files for disaster recovery purposes, and the DR system becomes active, replacing the primary system as the single source of truth for the application.

If you are consuming the data from client applications and need write access, we recommend providing that access via a RESTful web services API using the Harmony Core framework.