Notification or Subscription?
Two mechanisms currently coexist within Depot for notifying of changes to objects:
Certain mechanisms capture only some of the events, depending on the API used to drive Depot, and may offer different timing and event selection mechanisms.
The following places can be configured to requests notifications or subscriptions:
- on a Snowflake Location, the
enableEventsoption turns on Change Capture - on a Dataset:
- the
notificationsfield controls whether notifications are sent when the Transaction API is used to alter the dataset.- transaction started/completed events are always sent
- row-level change events are sent if the Change Capture has been enabled on the Location (Snowflake only)
- the
subscriptionsfield controls whether notifications are sent when select entities of the dataset are altered according to conditions specified in the subscription (works on all backends but only on Unitary APIs)
- the
The following table summarises the events that are captured and where they are sent:
| Output ⬇ / API mechanism ➡ | GraphQL, REST, Lambda | Transactions API | non-Depot mutations (direct sql changes, stored procedure, etc.) |
|---|---|---|---|
subscriptions (not SnowflakeLocation.enableEvents) | object-level changes Created/Updated/Deleted | NO | NO |
subscriptions (SnowflakeLocation.enableEvents == true) | NO (SUPPRESSED) | NO | NO |
notifications (not SnowflakeLocation.enableEvents) | no | transaction-level (immediate) | NO |
notifications (SnowflakeLocation.enableEvents == true) | row-level (DEFERRED) | transaction-level (immediate), row-level (deferred) | row-level only (deferred) |
On a Snowflake backend
When the Location.enableEvents option is enabled, Depot will use the Snowflake Change Capture mechanism (using streams)
in order to capture changes.
The streams are processed on a hourly basis.
In order to avoid duplicate events, the real-time events part of the subscription are turned off if the Location.enableEvents option is enabled.