Testing Migrations

Testing a migration consists of:

1. loading a previous version of the schema (e.g. the version currently in production, or the last version published before the implementation of the schema changes started)
2. setting up data conforming to that previous version
3. ordering Depot to update the database to the current version of the schema
4. testing the shape of the data after the migration

The before namespace for migrations should be retrieved from an older version of the package, whereas the current namespace is the after state of the migration. An older version of the package can be retrieved using npm/Yarn aliases, e.g. by adding the following to package.json:

"devDependencies": {
    "@stage-tech/my-pkg-2.12.1": "npm:@stage-tech/my-pkg@2.12.1",
}

Note that if the schema under test has a further version bump, the test should be updated so that after namespace also points to a fixed version of the package (or and additional migration added to cover the new version).

If all the environments get updated with the migration, the test itself is no longer valuable and could be removed, as the migration will not be used anymore.

See Migration/Transform DSL for more information on the format of the migrations document, this is optional and if not provided then only default/automatic migrations will be applied.

DepotTest-based testing

A migration is an operation available within the DepotTest object:

import { DepotTest } from "@stage-tech/depot-test";

import * as afterPkg from "..";
import * as beforePkg from "@stage-tech/my-pkg-2.12.1";

describe("some clever test name", () => {
    it("some clever scenario name", async () => {
        await DepotTest.in(beforePkg.mergedSchemas)
            .setContent('test/clever/name/data')
            .migrateTo({
                namespace: afterPkg.mergedSchemas,
                migrations: afterPkg.migrations,
            })
            .check({
                expectedUri: "test/clever/name/expected",
                actual: ["some.qualified.schema.name"],
            })
            .run();
    });
});

Testing Materialized View Migrations

Since materialized view migration has to be enabled per dataset and is not available as default, the naive approach of running the migrations will result in an error. DepotTest supports passing a flag in migrateTo method to enable this functionality. Note that to test it properly, materialized views should have seeded and expected data.

import { DepotTest } from "@stage-tech/depot-test";

import * as afterPkg from "..";
import * as beforePkg from "@stage-tech/my-pkg-2.12.1";

describe("some clever test name", () => {
    it("some clever scenario name", async () => {
        await DepotTest.in(beforePkg.mergedSchemas)
            .setContent('sample/materialized/views/data')
            .migrateTo({
                namespace: afterPkg.mergedSchemas,
                migrations: afterPkg.migrations,
                migrateMaterializedViews: true
            })
            .check({
                expectedUri: "sample/materialized/views/expected",
                actual: ["some.qualified.schema.name"],
            })
            .run();
    });
});

Legacy TestStep interface: Run a migration before or after a test step

caution

The toPassSchemaTest matcher (and the @stage-tech/depot-test/jest entry that registers it) is deprecated. For new code, use DepotTest.in(…).migrateTo(…).run() as shown above.

A migration can be run before or after the test step:

example.test.ts

// Deprecated: requires the Jest matchers entry. New code should use DepotTest.run().
import "@stage-tech/depot-test/jest";
import { TestStep } from "@stage-tech/depot-test";

import * as currentPkg from "..";
import * as beforePkg from "@stage-tech/my-pkg-2.12.1";

//collect all schemas used in package
const afterNamespace = JSON.stringify(currentPkg.mergedSchemas, null, 2);
const beforeNamespace = JSON.stringify(beforePkg.mergedSchemas, null, 2);
const migrations = JSON.stringify(currentPkg.migrations, null, 2);

describe("some clever test name", () => {
    it("some clever scenario name", () => {
        const steps: TestStep[] = [
            {
                name: "seed data",
                operation: Operation.UPSERT,
                dataUri: "test/clever/name/data"
            },
            {
                name: "migrate and assert data is populated",
                expectedUri: "test/clever/name/expected",
                actual: ["some.qualified.schema.name"],
                migrateBefore: {
                    migrationId: "my unique migration id",
                    namespace: afterNamespace,
                    migrations: migrations
                }
            }
        ];
        expect(steps).toPassSchemaTest(beforeNamespace);
    });
});

Migrations applied in a given step will be in force for the rest of the test.