authoring-dags

Name: authoring-dags
Author: astronomer

976installs55forks

How to install authoring-dags?

astronomer/agents/authoring-dags

$npx -y skills add astronomer/agents --skill authoring-dags

Installs into the current project.

Summary

Workflow and best practices for writing Apache Airflow DAGs. Use when creating a new DAG, write pipeline code, handling questions about DAG patterns and conventions or extending an existing DAG with a follow-up/downstream task. ANY request shaped like 'add a DAG named X', 'write

Files · 1

View on GitHub

SKILL.md

1	# DAG Authoring Skill
2
3	This skill guides you through creating and validating Airflow DAGs using best practices and `af` CLI commands.
4
5	> For testing and debugging DAGs, see the testing-dags skill which covers the full test -> debug -> fix -> retest workflow.
6
7	---
8
9	## Running the CLI
10
11	These commands assume `af` is on PATH. Run via `astro otto` to get it automatically, or install standalone with `uv tool install astro-airflow-mcp`.
12
13	---
14
15	## Workflow Overview
16
17	```
18	+-----------------------------------------+
19	\| 1. DISCOVER \|
20	\| Understand codebase & environment \|
21	+-----------------------------------------+
22	\|
23	+-----------------------------------------+
24	\| 2. PLAN \|
25	\| Propose structure, get approval \|
26	+-----------------------------------------+
27	\|
28	+-----------------------------------------+
29	\| 3. IMPLEMENT \|
30	\| Write DAG following patterns \|
31	+-----------------------------------------+
32	\|
33	+-----------------------------------------+
34	\| 4. VALIDATE \|
35	\| Check import errors, warnings \|
36	+-----------------------------------------+
37	\|
38	+-----------------------------------------+
39	\| 5. TEST (with user consent) \|
40	\| Trigger, monitor, check logs \|
41	+-----------------------------------------+
42	\|
43	+-----------------------------------------+
44	\| 6. ITERATE \|
45	\| Fix issues, re-validate \|
46	+-----------------------------------------+
47	```
48
49	---
50
51	## Phase 1: Discover
52
53	Before writing code, understand the context.
54
55	### Explore the Codebase
56
57	Use file tools to find existing patterns:
58	- `Glob` for `/dags//*.py` to find existing DAGs
59	- `Read` similar DAGs to understand conventions
60	- Check `requirements.txt` for available packages
61
62	### Query the Airflow Environment
63
64	Use `af` CLI commands to understand what's available:
65
66	\| Command \| Purpose \|
67	\|---------\|---------\|
68	\| `af config connections` \| What external systems are configured \|
69	\| `af config variables` \| What configuration values exist \|
70	\| `af config providers` \| What operator packages are installed \|
71	\| `af config version` \| Version constraints and features \|
72	\| `af dags list` \| Existing DAGs and naming conventions \|
73	\| `af config pools` \| Resource pools for concurrency \|
74
75	Example discovery questions:
76	- "Is there a Snowflake connection?" -> `af config connections`
77	- "What Airflow version?" -> `af config version`
78	- "Are S3 operators available?" -> `af config providers`
79
80	---
81
82	## Phase 2: Plan
83
84	Based on discovery, propose:
85
86	1. DAG structure - Tasks, dependencies, schedule
87	2. Operators to use - Based on available providers
88	3. Connections needed - Existing or to be created
89	4. Variables needed - Existing or to be created
90	5. Packages needed - Additions to requirements.txt
91
92	Get user approval before implementing.
93
94	---
95
96	## Phase 3: Implement
97
98	Write the DAG following best practices (see below). Key steps:
99
100	1. Create DAG file in appropriate location
101	2. Update `requirements.txt` if needed
102	3. Save the file
103
104	---
105
106	## Phase 4: Validate
107
108	Use `af` CLI as a feedback loop to validate your DAG.
109
110	### Step 1: Check Import Errors
111
112	After saving, check for parse errors (Airflow will have already parsed the file):
113
114	```bash
115	af dags errors
116	```
117
118	- If your file appears -> fix and retry
119	- If no errors -> continue
120
121	Common causes: missing imports, syntax errors, missing packages.
122
123	### Step 2: Verify DAG Exists
124
125	```bash
126	af dags get <dag_id>
127	```
128
129	Check: DAG exists, schedule correct, tags set, paused status.
130
131	### Step 3: Check Warnings
132
133	```bash
134	af dags warnings
135	```
136
137	Look for deprecation warnings or configuration issues.
138
139	### Step 4: Explore DAG Structure
140
141	```bash
142	af dags explore <dag_id>
143	```
144
145	Returns in one call: metadata, tasks, dependencies, source code.
146
147	### On Astro
148
149	If you're running on Astro, you can also validate locally before deploying:
150
151	- Parse check: Run `astro dev parse` to catch import errors and DAG-level issues without starting a full Airflow environment
152	- DAG-only deploy: Once validated, use `astro deploy --dags` for fast DAG-only deploys that skip the Docker image build — ideal for iterating on DAG code
153
154	---
155
156	## Phase 5: Test
157
158	> See the testing-dags skill for comprehensive testing guidance.
159
160	Once va

People who liked this also viewed

6 picks

Type

authoring-dags

976installs55forks

View on GitHub

How to install authoring-dags?

astronomer/agents/authoring-dags

$npx -y skills add astronomer/agents --skill authoring-dags

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# DAG Authoring Skill
2
3	This skill guides you through creating and validating Airflow DAGs using best practices and `af` CLI commands.
4
5	> For testing and debugging DAGs, see the testing-dags skill which covers the full test -> debug -> fix -> retest workflow.
6
7	---
8
9	## Running the CLI
10
11	These commands assume `af` is on PATH. Run via `astro otto` to get it automatically, or install standalone with `uv tool install astro-airflow-mcp`.
12
13	---
14
15	## Workflow Overview
16
17	```
18	+-----------------------------------------+
19	\| 1. DISCOVER \|
20	\| Understand codebase & environment \|
21	+-----------------------------------------+
22	\|
23	+-----------------------------------------+
24	\| 2. PLAN \|
25	\| Propose structure, get approval \|
26	+-----------------------------------------+
27	\|
28	+-----------------------------------------+
29	\| 3. IMPLEMENT \|
30	\| Write DAG following patterns \|
31	+-----------------------------------------+
32	\|
33	+-----------------------------------------+
34	\| 4. VALIDATE \|
35	\| Check import errors, warnings \|
36	+-----------------------------------------+
37	\|
38	+-----------------------------------------+
39	\| 5. TEST (with user consent) \|
40	\| Trigger, monitor, check logs \|
41	+-----------------------------------------+
42	\|
43	+-----------------------------------------+
44	\| 6. ITERATE \|
45	\| Fix issues, re-validate \|
46	+-----------------------------------------+
47	```
48
49	---
50
51	## Phase 1: Discover
52
53	Before writing code, understand the context.
54
55	### Explore the Codebase
56
57	Use file tools to find existing patterns:
58	- `Glob` for `/dags//*.py` to find existing DAGs
59	- `Read` similar DAGs to understand conventions
60	- Check `requirements.txt` for available packages
61
62	### Query the Airflow Environment
63
64	Use `af` CLI commands to understand what's available:
65
66	\| Command \| Purpose \|
67	\|---------\|---------\|
68	\| `af config connections` \| What external systems are configured \|
69	\| `af config variables` \| What configuration values exist \|
70	\| `af config providers` \| What operator packages are installed \|
71	\| `af config version` \| Version constraints and features \|
72	\| `af dags list` \| Existing DAGs and naming conventions \|
73	\| `af config pools` \| Resource pools for concurrency \|
74
75	Example discovery questions:
76	- "Is there a Snowflake connection?" -> `af config connections`
77	- "What Airflow version?" -> `af config version`
78	- "Are S3 operators available?" -> `af config providers`
79
80	---
81
82	## Phase 2: Plan
83
84	Based on discovery, propose:
85
86	1. DAG structure - Tasks, dependencies, schedule
87	2. Operators to use - Based on available providers
88	3. Connections needed - Existing or to be created
89	4. Variables needed - Existing or to be created
90	5. Packages needed - Additions to requirements.txt
91
92	Get user approval before implementing.
93
94	---
95
96	## Phase 3: Implement
97
98	Write the DAG following best practices (see below). Key steps:
99
100	1. Create DAG file in appropriate location
101	2. Update `requirements.txt` if needed
102	3. Save the file
103
104	---
105
106	## Phase 4: Validate
107
108	Use `af` CLI as a feedback loop to validate your DAG.
109
110	### Step 1: Check Import Errors
111
112	After saving, check for parse errors (Airflow will have already parsed the file):
113
114	```bash
115	af dags errors
116	```
117
118	- If your file appears -> fix and retry
119	- If no errors -> continue
120
121	Common causes: missing imports, syntax errors, missing packages.
122
123	### Step 2: Verify DAG Exists
124
125	```bash
126	af dags get <dag_id>
127	```
128
129	Check: DAG exists, schedule correct, tags set, paused status.
130
131	### Step 3: Check Warnings
132
133	```bash
134	af dags warnings
135	```
136
137	Look for deprecation warnings or configuration issues.
138
139	### Step 4: Explore DAG Structure
140
141	```bash
142	af dags explore <dag_id>
143	```
144
145	Returns in one call: metadata, tasks, dependencies, source code.
146
147	### On Astro
148
149	If you're running on Astro, you can also validate locally before deploying:
150
151	- Parse check: Run `astro dev parse` to catch import errors and DAG-level issues without starting a full Airflow environment
152	- DAG-only deploy: Once validated, use `astro deploy --dags` for fast DAG-only deploys that skip the Docker image build — ideal for iterating on DAG code
153
154	---
155
156	## Phase 5: Test
157
158	> See the testing-dags skill for comprehensive testing guidance.
159
160	Once va

People who liked this also viewed

6 picks

Type