Dataset Migrations

Overview

When the package inside a Dataset is updated, schemas may be added, changed or removed. The specific changes may be automatically inferred by comparing the schemas in the old and the new package version, or the user can provide an explicit migration 'hint' to help perform the necessary adjustments

Migration parameters at the dataset level can be configured in the Dataset object properties:

const myDataset = new depot.Dataset(this, 'MyDataset', {
  environment: depotEnvironment,
  package: dogSourcePackage,
  name: 'my-dataset',
  location: someSqlLocation,
  // executors...
  // additionalSchemas...
  // seeding...
  migrationOptions: {
    migrationMode: {
      type: 'asynchronous' as const,
      eventBusArn: customBus.eventBusArn,
    },
  },
});

The migrationOptions structure controls the migration process at a Dataset level. The following properties are available:

Property	Type	Default	Description
`disabled?`	`boolean`	false	Escape hatch, to disable Depot's schema version management entirely for this dataset
`migrateMaterializedViews`	`boolean`	false	should materialized views be migrated in place (`true`) or deleted and rebuilt at the next REFRESH operation? (`false`)
`triggerSeeding`	`boolean`	false	should seeding be triggered automatically after the migration process is complete?
`migrationMode`	`object`	see below	The migration mode to use for this dataset. (see below)

tip

triggerSeeding and migrationMode are new as of Depot 9.4.0

The migrationMode structure controls the choreography. Two struct types are available:

Synchronous migration (historic default)

Property	Type	Default	Description
`type`	`synchronous`		Migrations shall be executed synchronously during the deployment process.

Asynchronous migration (new as of Depot 9.4.0)

Property	Type	Default	Description
`type`	`asynchronous`		Migrations shall be executed asynchronously after the deployment process.
`eventBusArn`	`string`	default event bus	The ARN of the event bus to use to send the message about the deployment process.
`customEventBinding`	`boolean`	`false`	if `true`, the caller takes responsibility for handling migration messages themselves

Overview​

Overview