Legacy: TestStep-based tests

caution

This is the previous interface of depot-test prior to version 5.3.0. It is still available, and currently still underpins the DepotTest interface, but you may want to avoid using it in the future. You can follow this page for an upgrade guide.

The @stage-tech/depot-test/jest matchers entry used in the example below (expect(steps).toPassSchemaTest(…)) is itself deprecated and is Jest-only. New code should use DepotTest.in(…).….run() directly — see the upgrade guide.

Test step

The framework uses the testStep concept. Tests can have one or more steps. Each step can perform either a snowflake transaction or verify the state of the database, or both. The only restriction is that the first step must perform a transaction, otherwise schemas will not be created.

teststep-interface.ts

export interface TestStep {
    name?: string;
    operation?: Operation;
    dataUri?: string;
    source?: string;
    target?: string;
    query?: string;
    arguments?: { [key: string]: string };
    expectedUri?: string;
    assertions?: string[];
    createAssertionFiles?: boolean;
    skipExpressions?: boolean;
    stage?: Stage;
    migrateBefore?: Migration;
    migrateAfter?: Migration;
}

export interface Migration {
    migrationId: string;
    namespace: any;
    migrations?: any;
    migrationTimestamp?: Date;
}

export interface Stage {
    name: string;
    url: string;
    format?: string;
}

name is printed in logs at the beginning of each step execution.

Data action

operation, dataUri, expectedUri, source, target, query, arguments properties directly map to a Snowflake transaction.

stage is used to override external stages with a local stage for testing purposes. name must be qualified name of the external stage that you want to override, url is a data file location that will be loaded into a stage, format is a way to provide file format (by fully qualified name) that will be used to load data into a stage. The default format is JSON.

warning

stage override functionality requires a snowflake secrets manager credentials ARN to be present in configuration

Data files

Data loading step looks for file or folder that matches dataUri/target i.e. dataUri=src/test/data/ and target=pet.Store step will look and try to load either a file src/test/data/pet.Store.json or all files in src/test/data/pet.Store/.

Data assertions

expectedUri and assertions go together and are used in the verification process. Each file or folder in the expectedUri folder that matches schema names provided in assertions will be compared against full data from a table with the same view.

createAssertionFiles is a property that is enabled by default and creates (if not exists) assertion files from assertions list and fill them with data from corresponding tables.

Supported formats

• JSON - .json with array of objects inside
• JSONL - .jsonl JSON lines is default snowflake native format
• CSV - .csv coma separated with header and comments lines started with #
• HOCON - .conf data will be loaded from data parameter that must contain array of objects. Read more about HOCON

Getting started using TestStep

Simple example:

example.test.ts

// import the Jest bindings
import "@stage-tech/depot-test/jest";
import { TestStep } from "@stage-tech/depot-test";

import * as pkg from "..";

//collect all schemas used in package
const namespace = JSON.stringify(pkg.mergedSchemas, null, 2);

describe("some clever test name", () => {
    it("some clever scenario name", () => {
        const step: TestStep = {
            name: "contact identities view is populated",
            operation: Operation.UPSERT,
            dataUri: "test/clever/name/data",
            expectedUri: "test/clever/name/expected",
            assertions: ["some.qualified.schema.name"]
        };
        const steps: Array<TestStep> = [step];
        expect(steps).toPassSchemaTest(namespace);
    });
});