Skip to main content

Cross-Location sync

Depot provides support for transactions that replicate data across multiple, technologically separate locations. This feature enables seamless data synchronization between locations, ensuring data integrity and consistency across distinct environments.

Overview​

Cross-Location Transactions: Depot’s transaction replication operates as a cross-location transaction system, allowing for the synchronized movement of data between different storage engines.

warning

SYNC operation doesn't track any state or do any data validation this responsibility is left to the user of SYNC operation.

Source Schema Requirements​

For successful replication, the source schema must meet the following conditions:

History Enabled: The source schema must have history tracking enabled to allow Depot to capture and replay changes accurately over time. This ensures that any updates, deletions, or insertions in the source can be consistently applied to the target location.

Column Matching: The source schema should include all required columns defined in the target schema. This alignment ensures that data integrity is maintained and no critical data elements are lost or misaligned during replication.

Example Transaction​

Below is an example of initiating a cross-location transaction in Depot:

const tr: TransactionRequest.Action = {
  source: 'DogSource',
  operation: TransactionRequest.Operation.SYNC,
  target: 'DogTarget',
  sourceDatasetId: env.datasetId,
  arguments: { from: fromTime, to: toTime }
};

Key moments:

  • provide sourceDatasetId that is in different storage engine
  • provide arguments { from: Date | string, to: Date | string }

Best Practices​

Ensure Schema Consistency:

  • Double-check that all required columns in the target schema are present in the source schema before initiating replication.
  • Verify Date Format Compliance: Use a consistent date format across transactions (Date object or standard date string) to avoid errors.