Skip to content

Target Systems

Target systems are systems that receive the replicated data in some form or that process the stream of CDC messages for some other purpose. The Streaming Integration Platform currently includes pre-built solutions for two target system use cases: data replication to SDMS files and data replication to Microsoft SQL Server.

Data Replication to SDMS Files

The ISAM Agent can be deployed to replicate SDMS or RMS ISAM and relative data files to a duplicate set of files, typically on a different system.

The primary use cases for doing so include

  • Phased migration to a new platform
  • Offloading workloads from the system of record to other systems
  • Disaster recovery solutions

Data Replication to Microsoft SQL Server

The SQL Agent can be deployed to replicate SDMS or RMS ISAM and relative data to a Microsoft SQL Server database.

The primary use cases for doing so include

  • Exposing the data in a form easily consumable by other home-grown or third-party applications, without any significant impact on the system of record
  • Data warehousing or other ETL use cases

Latency

The Streaming Integration Platform is designed to achieve high performance and throughput, resulting in the replication of data in near real time. But regardless of how small the changes are, there is always some amount of latency between changes to the system-of-record data and the replicated data, and this latency can be affected by the choice of technologies and deployment environments.

The environment is by definition distributed across multiple systems, usually involving at least three different system environments:

  • The system of record
  • The Kafka (or Redpanda) environment
  • One or more target systems

Ideally the network connections between these environments should be as fast and low latency as possible, but in some cases some or all of these components may be physically separated, and perhaps even cloud based. The latency and throughput of the overall solution will depend heavily on these factors.

Read-Only Access

In all scenarios, the replicated data should be treated as a read-only data source. The one exception to that rule is if the data has been replicated to SDMS files for disaster recovery purposes, and the DR system becomes active, replacing the primary system as the single source of truth for the application.

If you are consuming the data from client applications and need write access, we recommend providing that access via a RESTful web services API using the Harmony Core framework.