Migrations

To migrate between schema versions automatically, you should start to use versions for significant changes to schemas.

The schema version is an integer that defaults to 1. When you want to change a schema and apply migration automatically, the version of the schema should be incremented to the next number.

A migration document can be supplied alongside a deployment containing additional transformations where the migration to apply cannot be inferred automatically. For more information see Expression Migrations or Raw SQL Migrations

For more information on how to test migrations in order to avoid deployment failures, see the Testing Migrations documentation.

To supply migrations during a deployment, a set of migration files (in YAML or as typescript) should be placed into a folder within a package repository, or supplied as typescript objects:

import { Schemas, Migrations } from "@stage-tech/depot-package-common";
const schemas = Schemas.fromYamlFiles(["./schemas/my.contacts", "./schemas/my.addresses"]);
const migrations = Migrations.fromYamlFiles(["./migrations"]);
const myPackage = new Package(this, "MyPackage", {
    name: "my.package",
    version: "1.0.0",
    schemas: schemas,
    migrations: migrations
});

const myDataset = new Dataset(stack, "MyDataset", {
    environment: myEnv,
    package: myPackage,
    ...etc...
});

In the /migrations folder, multiple migrations documents may be provided, all will be supplied to the deployment of a given dataset, but only those that relate to version changes in the current deployment will be applied.

For example, if the current deployed package is:

SchemaA:
  type: object
  version: 1
  properties:
    field1:
      type: string

SchemaB:
  type: object
  version: 3
  properties:
    field1:
      type: string

And the new version of the package is:

SchemaA:
  type: object
  version: 2
  properties:
    field1:
      type: string

SchemaB:
  type: object
  version: 4
  properties:
    field1:
      type: string

And the following transform documents are provided:

/migrations/SchemaA/2.yml

- target: SchemaA
  fromVersion: 1
  toVersion: 2

/migrations/SchemaB/3.yml

- target: SchemaA
  fromVersion: 2
  toVersion: 3

Then the migration provided for SchemaA will be applied, but the migration for SchemaB will not, which may or may not be an error condition, depending on whether the migration from version 3 to version 4 can be applied automatically.

Migrations are traditionally executed 'synchronously', i.e. as part of the deployment. As part of Depot 9.4.0, it is possible to execute migrations 'asynchronously', i.e. as a separate step after the deployment. More details are available as part of the Dataset Migrations in CDK documentation.