Skip to main content

Snowflake Transactions

Snowflake Depot Transactions use the same schema as standard Depot Transactions. This page exists currently to help list good to know information and “gotchas” about the format of Data and how Snowflake Transactions work.

Depot-Snowflake can participate in Cross-Location Transactions.

Data formats​

Input data for a Transaction (specified by the locationUri of a Transaction action) can have different column casing ( compared to the schema definition). Use the column formats functionality to configure this in your Transactions.

Required “id” field​

When ingesting data into object schemas, the id field is mandatory and the data cannot have null values in the id field. Note that the id field is implicit in object schemas and is not explicitly defined in the schema definition.

Creating Transactions​

All other Depot Transaction creation methods are valid for Snowflake Transactions. For quick reference, here is how you would create a Transaction using the Depot CLI:

depot create-transaction --dataset 282d8404e895 --executor 871efk4421e6b --data "s3://example-ingest-data/demo-load-1" --format JSON

To create a Transaction using the Depot Lambda Gateway directly, you would invoke the Lambda function with a payload like this:

{
  "schema": "Transaction",
  "operation": "create",
  "data": {
    "dataset": {
      "id": "dataset123"
    },
    "actions": [
      {
        "locationUri": "s3://my-data-example-bucket/path",
        "operation": "UPSERT"
      }
    ],
    "context": {
      "trace": "xx1"
    }
  }
}
info

The dataset.id argument refers relevant dataset resource in your environment. When using a Snowflake Location for your target Dataset, that Snowflake Database should also have an external Stage configured that allows access to your data specified in the locationUri. (Using the createEntities = true flag for your Location will set this up automatically).