Aurora

Intro

Depot-Aurora location is a wrapper over AWS Aurora service with postgres(15.3) under the hood. How to add Depot-Aurora to your service can be found here.

Database base structure

All datasets that uses same cluster will be stored in same database named depotpostgres. Aurora uses lower_snake casing for database schemas and entity names. All entities from the dataset will be stored in single database schema matching dataset name. All depot entities such as history tables and shadow views will end with double underscore and the purpose of the entity i.e. pet_store__history

warning

Postgres has a default limitation of 63 bytes for database names and 63 bytes for schema names. These limitations cannot be exceeded on Aurora RDS for Postgres, which we're using.

If a Basestar (Depot) entity name or field name leads to a table, view or column identifier longer than 63 characters, it will be rejected and not provisioned.

On system-derived entity names (such as "former names" for column and object soft deletes), Depot may use combination of truncation and hashing strategies to ensure that the names are within the 63-character limit.

For entity names directly derived from names you supply, you must ensure that the names are within the 63-character limit, or use the extensions system (see below) to supply alternative physical names for the entities.

Capabilities

CRUD

Depot-Aurora provides fast response times over rest and Graphql. Cross-dataset joins are supported at the SQL level if datasets exist in the same Aurora cluster.

Depot transactions

Depot-Aurora supports all transaction types between entities, imports and exports from CSV (with some limitations), imports and exports from JSON Lines (JSONL), with some limitations.

Depot-Aurora can participate in Cross-Location Transactions.

Not supported

Depot aurora transactions don't support udf's and procedures.

Depot import/export don't support Parquet, Orc or Avro formats.

Composite support

Depot-aurora only supports sync, with plans to support async replication.

Extensions

In case a Depot schema name is too long to fit into the 63 character identifier limit, you can override Depot's naming policy and force a shorter name, using the sql.table extension:

my.ObjectWithAVeryLongAndInconvenientName:
  type: object
  properties:
    id:
      type: string
      primary: true
    name:
      type: string
  extensions:
    sql.table: my_schema.my_object

With the last clause, instead of deploying to a table tentatively named my_object_with_a_very_long_and_inconvenient_name, Depot will deploy into a table named my_object in the schema my_schema.

It will also deploy a table named my_object__history in the same schema, for history tracking.

The sql.history.table extension can be used to override the history table name, if the default policy of adding the __history suffix is unsuitable.

tip

The Postgres table limit is 63 characters. If your object table requires history, the default Depot policy of adding __history constrains your custom table name to 54 characters. That limitation is lifted if your table has history disabled, or if you provide your own custom name for the history table as well.

tip

The Snowflake Locations also respond to sql.table and sql.history.table extensions. If your schemas need to be deployable on both types (e.g. you use a "Test Locally on Postgres, Deploy on Snowflake" strategy), then you may need to use different names.

You can use aurora.sql.table and aurora.sql.history.table as alternate extensions only Aurora responds to.