Adaptive dbt Routing

USE CASE

Adaptive dbt Routing

Getting dbt to complete quickly and efficiently is often a key concern of data organizations. Using Sundeck's advanced capabilities, you can enable per-model routing in just a few simple steps, maximizing parallelization and throughput while minimizing waste.

Step 1: Tag your queries with dbt comments

Use either the built in query-comments or the more advanced dbt-snowflake-monitoring package to capture information about your dbt runs in query history.

Step 2: Setup Routing Analysis Task & Table

Install our simple analysis task to analyze past dbt operations and start maintaining a routing table for future dbt runs.

Step 3: Use SQL Pre-hook Routing Table Lookup

Sundeck comes pre-installed with a hook designed specifically to inspect query comments and reroute operations based on them. Start using this connection to optimize dbt behavior.

Step 4: Profit!

Future dbt runs will now benefit from dynamic routing decisions. The ongoing task will make adjustments based on past dbt model executions, optimizing your situation.