The orderSchemasByUsingClause() utility function

The orderSchemasByUsingClause() utility function (exported by "@stage-tech/depot-package-common") facilitates the navigation in a graph of interdependent schemas

The main use case for this is the definition of package transactions in a robust way, that remains independent of minor schema additions or fails fast in testing should a breaking change be accidentally introduced.

Example: safely refreshing a graph of materialized views

Assuming your schema package currently contains the following schemas:

In which order should you refresh the materialized views? Glancing at the graph gives us immediately that we should refresh MatView2 and MatView3, then MatView5 and MatView6 and finally MatView7.

Unfortunately, it isn't that obvious to identify this from the source Yaml representation's using definition, especially if the task is not initial creation, but addition into a possibly unfamiliar graph during a later iteration.

The orderSchemasByUsingClause() utility function facilitates the navigation into the dependency graph in a way that will highlight breaking changes, guarantee the absence of cycles, and provide a stable and robust visit order.

Example package-transactions.ts file:

import { TransactionRequest, orderSchemasByUsingClause } from "@stage-tech/depot-package-common";

class DocPackageTransactions {
    public static refreshMatView7FromObj1(): TransactionRequest.Action[] {
        return orderSchemasByUsingClause(docPackage, [
          /* need to mention all the subgraph, in any order: */
            "Obj1",
            "MatView2",
            "MatView3",
            "MatView5",
            "MatView6",
            "View4",
            "MatView7"
        ])
            // extract the materialized views only, in the correct order:
            .flatMap(layer =>
                layer.schemaNames.filter(schemaName => {
                    const schema = docPackage[schemaName];
                    return isViewSchema(schema) && schema.materialized;
                })
            )
            // for each view, shape the correct refresh action
            .map(viewName => {
                return {
                    operation: TransactionRequest.Operation.REFRESH,
                    target: viewName
                    /* could add a `query` with e.g. ASSET_ID parameters here if needed */
                };
            });
    }
}

As long as DocPackageTransactions.refreshMatView7FromObj1() is exercised in the schema package's test suite (see package transactions for an example), it is impossible to inadvertently introduce a table, view or materialized view in the dependencies of MatView7 without noticing at the latest during CI.

Here the output of DocPackageTransactions.refreshMatView7FromObj1() is

[
    { "operation": "REFRESH", "target": "MatView2" },
    { "operation": "REFRESH", "target": "MatView3" },
    { "operation": "REFRESH", "target": "MatView5" },
    { "operation": "REFRESH", "target": "MatView6" },
    { "operation": "REFRESH", "target": "MatView7" }
]

which, if played as a transaction, guarantees that MatView7 will be refreshed after all its dependencies all the way to the source Obj1.