cosmos-dbt-fusion

Name: cosmos-dbt-fusion
Author: astronomer

748installs55forks

How to install cosmos-dbt-fusion?

astronomer/agents/cosmos-dbt-fusion

$npx -y skills add astronomer/agents --skill cosmos-dbt-fusion

Installs into the current project.

Summary

Run a dbt Fusion project with Astronomer Cosmos. Use when running a dbt Fusion project with Astronomer Cosmos (Cosmos 1.11+, ExecutionMode.LOCAL on Snowflake/Databricks). Before implementing, verify dbt engine is Fusion (not Core), the warehouse is supported, and local execution

Files · 1

View on GitHub

SKILL.md

1	# Cosmos + dbt Fusion: Implementation Checklist
2
3	Execute steps in order. This skill covers Fusion-specific constraints only.
4
5	> Version note: dbt Fusion support was introduced in Cosmos 1.11.0. Requires Cosmos ≥1.11.
6	>
7	> Reference: See [reference/cosmos-config.md](reference/cosmos-config.md) for ProfileConfig, operator_args, and Airflow 3 compatibility details.
8
9	> Before starting, confirm: (1) dbt engine = Fusion (not Core → use cosmos-dbt-core), (2) warehouse = Snowflake, Databricks, Bigquery and Redshift only.
10
11	### Fusion-Specific Constraints
12
13	\| Constraint \| Details \|
14	\|------------\|---------\|
15	\| No async \| `AIRFLOW_ASYNC` not supported \|
16	\| No virtualenv \| Fusion is a binary, not a Python package \|
17	\| Warehouse support \| Snowflake, Databricks, Bigquery and Redshift support [while in preview](https://github.com/dbt-labs/dbt-fusion) \|
18
19	---
20
21	## 1. Confirm Cosmos Version
22
23	> CRITICAL: Cosmos 1.11.0 introduced dbt Fusion compatibility.
24
25	```bash
26	# Check installed version
27	pip show astronomer-cosmos
28
29	# Install/upgrade if needed
30	pip install "astronomer-cosmos>=1.11.0"
31	```
32
33	Validate: `pip show astronomer-cosmos` reports version ≥ 1.11.0
34
35	---
36
37	## 2. Install the dbt Fusion Binary (REQUIRED)
38
39	dbt Fusion is NOT bundled with Cosmos or dbt Core. Install it into the Airflow runtime/image.
40
41	Determine where to install the Fusion binary (Dockerfile / base image / runtime).
42
43	### Example Dockerfile Install
44
45	```dockerfile
46	USER root
47	RUN apt-get update && apt-get install -y curl
48	ENV SHELL=/bin/bash
49	RUN curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh \| sh -s -- --update
50	USER astro
51	```
52
53	### Common Install Paths
54
55	\| Environment \| Typical path \|
56	\|-------------\|--------------\|
57	\| Astro Runtime \| `/home/astro/.local/bin/dbt` \|
58	\| System-wide \| `/usr/local/bin/dbt` \|
59
60	Validate: The `dbt` binary exists at the chosen path and `dbt --version` succeeds.
61
62	---
63
64	## 3. Choose Parsing Strategy (RenderConfig)
65
66	Parsing strategy is the same as dbt Core. Pick ONE:
67
68	\| Load mode \| When to use \| Required inputs \|
69	\|-----------\|-------------\|-----------------\|
70	\| `dbt_manifest` \| Large projects; fastest parsing \| `ProjectConfig.manifest_path` \|
71	\| `dbt_ls` \| Complex selectors; need dbt-native selection \| Fusion binary accessible to scheduler \|
72	\| `automatic` \| Simple setups; let Cosmos pick \| (none) \|
73
74	```python
75	from cosmos import RenderConfig, LoadMode
76
77	_render_config = RenderConfig(
78	load_method=LoadMode.AUTOMATIC, # or DBT_MANIFEST, DBT_LS
79	)
80	```
81
82	---
83
84	## 4. Configure Warehouse Connection (ProfileConfig)
85
86	> Reference: See [reference/cosmos-config.md](reference/cosmos-config.md#profileconfig-warehouse-connection) for full ProfileConfig options and examples.
87
88
89	```python
90	from cosmos import ProfileConfig
91	from cosmos.profiles import SnowflakeUserPasswordProfileMapping
92
93	_profile_config = ProfileConfig(
94	profile_name="default",
95	target_name="dev",
96	profile_mapping=SnowflakeUserPasswordProfileMapping(
97	conn_id="snowflake_default",
98	),
99	)
100	```
101
102	---
103
104	## 5. Configure ExecutionConfig (LOCAL Only)
105
106	> CRITICAL: dbt Fusion with Cosmos requires `ExecutionMode.LOCAL` with `dbt_executable_path` pointing to the Fusion binary.
107
108	```python
109	from cosmos import ExecutionConfig
110	from cosmos.constants import InvocationMode
111
112	_execution_config = ExecutionConfig(
113	invocation_mode=InvocationMode.SUBPROCESS,
114	dbt_executable_path="/home/astro/.local/bin/dbt", # REQUIRED: path to Fusion binary
115	# execution_mode is LOCAL by default - do not change
116	)
117	```
118
119	---
120
121	## 6. Configure Project (ProjectConfig)
122
123	```python
124	from cosmos import ProjectConfig
125
126	_project_config = ProjectConfig(
127	dbt_project_path="/path/to/dbt/project",
128	# manifest_path="/path/to/manifest.json", # for dbt_manifest load mode
129	# install_dbt_deps=False, # if deps precomputed in CI
130	)
131	```
132
133	---
134
135	## 7. Assemble DAG / TaskGroup
136
137	### Option A: DbtDag (Standalone)
138
139	```python
140	from cosmos import DbtDag, ProjectConfig, ProfileConfig, ExecutionConfig, RenderConfig
141	from cosmos.profiles import SnowflakeUserPasswordProfileMapping
142	from pendulum import datetime
143
144	_project_config = ProjectConfig(
145	dbt_project_path="/usr/local/airflow/dbt/my_project",
146	)
147
148	_profile_config = ProfileConfig(
149	profile_name="default",
150	target_name="dev",
151	profile_mapping=SnowflakeUserPasswordProfileMapping(
152	conn_id="snowflake_default",
153	),
154	)
155
156	_execution_config = ExecutionConfig(
157	dbt_executable_path="/home/astro/.local/bin/dbt", # Fusion binary
158	)
159
160	_render_config = RenderConfig()
161
162	my_fusion_dag = DbtDag(
163	dag_id="my_fusion_cosmos_dag",
164	project_config=_project_config,
165	profile_config=_pr

People who liked this also viewed

6 picks

Type

cosmos-dbt-fusion

748installs55forks

View on GitHub

How to install cosmos-dbt-fusion?

astronomer/agents/cosmos-dbt-fusion

$npx -y skills add astronomer/agents --skill cosmos-dbt-fusion

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Cosmos + dbt Fusion: Implementation Checklist
2
3	Execute steps in order. This skill covers Fusion-specific constraints only.
4
5	> Version note: dbt Fusion support was introduced in Cosmos 1.11.0. Requires Cosmos ≥1.11.
6	>
7	> Reference: See [reference/cosmos-config.md](reference/cosmos-config.md) for ProfileConfig, operator_args, and Airflow 3 compatibility details.
8
9	> Before starting, confirm: (1) dbt engine = Fusion (not Core → use cosmos-dbt-core), (2) warehouse = Snowflake, Databricks, Bigquery and Redshift only.
10
11	### Fusion-Specific Constraints
12
13	\| Constraint \| Details \|
14	\|------------\|---------\|
15	\| No async \| `AIRFLOW_ASYNC` not supported \|
16	\| No virtualenv \| Fusion is a binary, not a Python package \|
17	\| Warehouse support \| Snowflake, Databricks, Bigquery and Redshift support [while in preview](https://github.com/dbt-labs/dbt-fusion) \|
18
19	---
20
21	## 1. Confirm Cosmos Version
22
23	> CRITICAL: Cosmos 1.11.0 introduced dbt Fusion compatibility.
24
25	```bash
26	# Check installed version
27	pip show astronomer-cosmos
28
29	# Install/upgrade if needed
30	pip install "astronomer-cosmos>=1.11.0"
31	```
32
33	Validate: `pip show astronomer-cosmos` reports version ≥ 1.11.0
34
35	---
36
37	## 2. Install the dbt Fusion Binary (REQUIRED)
38
39	dbt Fusion is NOT bundled with Cosmos or dbt Core. Install it into the Airflow runtime/image.
40
41	Determine where to install the Fusion binary (Dockerfile / base image / runtime).
42
43	### Example Dockerfile Install
44
45	```dockerfile
46	USER root
47	RUN apt-get update && apt-get install -y curl
48	ENV SHELL=/bin/bash
49	RUN curl -fsSL https://public.cdn.getdbt.com/fs/install/install.sh \| sh -s -- --update
50	USER astro
51	```
52
53	### Common Install Paths
54
55	\| Environment \| Typical path \|
56	\|-------------\|--------------\|
57	\| Astro Runtime \| `/home/astro/.local/bin/dbt` \|
58	\| System-wide \| `/usr/local/bin/dbt` \|
59
60	Validate: The `dbt` binary exists at the chosen path and `dbt --version` succeeds.
61
62	---
63
64	## 3. Choose Parsing Strategy (RenderConfig)
65
66	Parsing strategy is the same as dbt Core. Pick ONE:
67
68	\| Load mode \| When to use \| Required inputs \|
69	\|-----------\|-------------\|-----------------\|
70	\| `dbt_manifest` \| Large projects; fastest parsing \| `ProjectConfig.manifest_path` \|
71	\| `dbt_ls` \| Complex selectors; need dbt-native selection \| Fusion binary accessible to scheduler \|
72	\| `automatic` \| Simple setups; let Cosmos pick \| (none) \|
73
74	```python
75	from cosmos import RenderConfig, LoadMode
76
77	_render_config = RenderConfig(
78	load_method=LoadMode.AUTOMATIC, # or DBT_MANIFEST, DBT_LS
79	)
80	```
81
82	---
83
84	## 4. Configure Warehouse Connection (ProfileConfig)
85
86	> Reference: See [reference/cosmos-config.md](reference/cosmos-config.md#profileconfig-warehouse-connection) for full ProfileConfig options and examples.
87
88
89	```python
90	from cosmos import ProfileConfig
91	from cosmos.profiles import SnowflakeUserPasswordProfileMapping
92
93	_profile_config = ProfileConfig(
94	profile_name="default",
95	target_name="dev",
96	profile_mapping=SnowflakeUserPasswordProfileMapping(
97	conn_id="snowflake_default",
98	),
99	)
100	```
101
102	---
103
104	## 5. Configure ExecutionConfig (LOCAL Only)
105
106	> CRITICAL: dbt Fusion with Cosmos requires `ExecutionMode.LOCAL` with `dbt_executable_path` pointing to the Fusion binary.
107
108	```python
109	from cosmos import ExecutionConfig
110	from cosmos.constants import InvocationMode
111
112	_execution_config = ExecutionConfig(
113	invocation_mode=InvocationMode.SUBPROCESS,
114	dbt_executable_path="/home/astro/.local/bin/dbt", # REQUIRED: path to Fusion binary
115	# execution_mode is LOCAL by default - do not change
116	)
117	```
118
119	---
120
121	## 6. Configure Project (ProjectConfig)
122
123	```python
124	from cosmos import ProjectConfig
125
126	_project_config = ProjectConfig(
127	dbt_project_path="/path/to/dbt/project",
128	# manifest_path="/path/to/manifest.json", # for dbt_manifest load mode
129	# install_dbt_deps=False, # if deps precomputed in CI
130	)
131	```
132
133	---
134
135	## 7. Assemble DAG / TaskGroup
136
137	### Option A: DbtDag (Standalone)
138
139	```python
140	from cosmos import DbtDag, ProjectConfig, ProfileConfig, ExecutionConfig, RenderConfig
141	from cosmos.profiles import SnowflakeUserPasswordProfileMapping
142	from pendulum import datetime
143
144	_project_config = ProjectConfig(
145	dbt_project_path="/usr/local/airflow/dbt/my_project",
146	)
147
148	_profile_config = ProfileConfig(
149	profile_name="default",
150	target_name="dev",
151	profile_mapping=SnowflakeUserPasswordProfileMapping(
152	conn_id="snowflake_default",
153	),
154	)
155
156	_execution_config = ExecutionConfig(
157	dbt_executable_path="/home/astro/.local/bin/dbt", # Fusion binary
158	)
159
160	_render_config = RenderConfig()
161
162	my_fusion_dag = DbtDag(
163	dag_id="my_fusion_cosmos_dag",
164	project_config=_project_config,
165	profile_config=_pr

People who liked this also viewed

6 picks

Type