migrating-dagster-to-airflow

Name: migrating-dagster-to-airflow
Author: astronomer

19installs55forks

How to install migrating-dagster-to-airflow?

astronomer/agents/migrating-dagster-to-airflow

$npx -y skills add astronomer/agents --skill migrating-dagster-to-airflow

Installs into the current project.

Summary

Guide for migrating Dagster projects to Apache Airflow 3 on Astro. Use when the user mentions migrating, converting, or porting Dagster (or Dagster+) code to Airflow or Astro, wants to plan or assess such a migration, or asks what a Dagster construct maps to in Airflow. Covers as

Files · 1

View on GitHub

SKILL.md

1	# Dagster → Airflow 3 (Astro) migration
2
3	Migrate a Dagster project to Airflow 3 on Astro Runtime, honestly. The migration is asset-first (Dagster asset graphs translate to Airflow assets and asset-aware schedules, not flattened DAGs), incremental (domain by domain, Dagster stays authoritative until parity), and honest (every definition gets an explicit disposition; semantic deltas are documented, never papered over).
4
5	First time driving this? Read `reference/quickstart.md` first: hour-one commands, the glossary, and what can and cannot break.
6
7	## Migration at a glance
8
9	1. Baseline the source project's tests, then inventory it read-only (`scripts/inventory.py` → manifest).
10	2. Review classifications (MECH/JUDG/REDESIGN/NONE per `reference/mapping.md`); make the go/no-go call (three outcomes; migrate-with-conditions is the common case, stay is the narrow one); plan DAG boundaries, per-edge IO decisions, and Gate 3 expectations into the manifest.
11	3. Trial-migrate 2-3 representative units end-to-end through every validation gate.
12	4. Migrate domain by domain through the six-gate ladder (`reference/validation.md`), tracking per-unit state (`scripts/status.py`); fix failure classes via `reference/troubleshooting.md`, never stub.
13	5. Map the platform layer (secrets, alerts, CI/CD, Deployments) per `reference/astro-deployment.md`.
14	6. Run side by side, then cut over per domain (consumers unpause first; see the checklist), keeping rollback one step away.
15	7. Deliver the migration report: every definition dispositioned, an equivalence row per trigger, losses stated plainly.
16
17	## Version drift
18
19	Verified against Airflow 3.3.0 / Astro Runtime 3.3-2 / astronomer-cosmos 1.15 / Dagster 1.13 (2026-07). Version-sensitive rows in the references carry their floor (notably the 3.2-vs-3.3 partition surface). Before relying on a version-gated claim: check the target (`airflow version`, `astro deployment inspect`), probe imports for sdk surface (`python3 -c "from airflow.sdk import X"`), and prefer `--help` / API spec discovery over assuming verbatim CLI/REST contracts on newer versions. Playbook entries are version-scoped per entry.
20
21	## Requirements
22
23	- Target Astro Runtime 3.3+ (Airflow 3.3+); the native asset-partition surface requires it. Below 3.2 the mapping degrades badly; say so and recommend upgrading before migrating.
24	- The Dagster repo, and ideally a running Dagster instance (its materialization metadata provides parity-test fixtures).
25	- `astro` CLI for the target project.
26
27	## Hard rules
28
29	1. Never stub. A translated unit either works through its validation gate or is deferred with a written reason. Fake-success bodies and workaround code with long justifying comments are failures.
30	2. No silent omissions. Every record in the inventory manifest ends `complete` or `deferred (reason)`. `scripts/status.py summary` exits nonzero otherwise; run it before claiming done.
31	3. Equivalence rows for every trigger. Each schedule/sensor/automation condition gets a report row: source spelling, target spelling, delta in one sentence. Semantic deltas exist (catchup, on_cron inversion, eager guarantees); the sin is not the delta, it is the undocumented delta.
32	4. Fix classes, not instances. When a translation pattern fails validation, fix the pattern (and record it in `reference/troubleshooting.md`), then re-apply; do not hand-patch one unit.
33	5. Do not invent APIs. The references contain verified names only. Anything not covered there gets verified against official docs before use.
34
35	## Workflow
36
37	### Phase 0: Preflight
38
39	Confirm target Runtime version, `astro` CLI presence, and repo access. Detect the project layout: classic (`@repository`/`workspace.yaml`), modern (`Definitions`), or Components (`pyproject.toml [tool.dg]`, `defs.yaml` files); all three occur, sometimes together. Baseline the source project's test suite now: pre-existing failures are recorded and excluded from migration blame.
40
41	### Phase 1: Inventory (read-only)
42
43	```
44	python3 scripts/inventory.py <dagster_repo> --out manifest.json # static scan
45	python3 scripts/inventory.py <dagster_repo> --runtime --out manifest.json # + runtime introspection when the project imports
46	`

People who liked this also viewed

6 picks

Type

migrating-dagster-to-airflow

19installs55forks

View on GitHub

How to install migrating-dagster-to-airflow?

astronomer/agents/migrating-dagster-to-airflow

$npx -y skills add astronomer/agents --skill migrating-dagster-to-airflow

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Dagster → Airflow 3 (Astro) migration
2
3	Migrate a Dagster project to Airflow 3 on Astro Runtime, honestly. The migration is asset-first (Dagster asset graphs translate to Airflow assets and asset-aware schedules, not flattened DAGs), incremental (domain by domain, Dagster stays authoritative until parity), and honest (every definition gets an explicit disposition; semantic deltas are documented, never papered over).
4
5	First time driving this? Read `reference/quickstart.md` first: hour-one commands, the glossary, and what can and cannot break.
6
7	## Migration at a glance
8
9	1. Baseline the source project's tests, then inventory it read-only (`scripts/inventory.py` → manifest).
10	2. Review classifications (MECH/JUDG/REDESIGN/NONE per `reference/mapping.md`); make the go/no-go call (three outcomes; migrate-with-conditions is the common case, stay is the narrow one); plan DAG boundaries, per-edge IO decisions, and Gate 3 expectations into the manifest.
11	3. Trial-migrate 2-3 representative units end-to-end through every validation gate.
12	4. Migrate domain by domain through the six-gate ladder (`reference/validation.md`), tracking per-unit state (`scripts/status.py`); fix failure classes via `reference/troubleshooting.md`, never stub.
13	5. Map the platform layer (secrets, alerts, CI/CD, Deployments) per `reference/astro-deployment.md`.
14	6. Run side by side, then cut over per domain (consumers unpause first; see the checklist), keeping rollback one step away.
15	7. Deliver the migration report: every definition dispositioned, an equivalence row per trigger, losses stated plainly.
16
17	## Version drift
18
19	Verified against Airflow 3.3.0 / Astro Runtime 3.3-2 / astronomer-cosmos 1.15 / Dagster 1.13 (2026-07). Version-sensitive rows in the references carry their floor (notably the 3.2-vs-3.3 partition surface). Before relying on a version-gated claim: check the target (`airflow version`, `astro deployment inspect`), probe imports for sdk surface (`python3 -c "from airflow.sdk import X"`), and prefer `--help` / API spec discovery over assuming verbatim CLI/REST contracts on newer versions. Playbook entries are version-scoped per entry.
20
21	## Requirements
22
23	- Target Astro Runtime 3.3+ (Airflow 3.3+); the native asset-partition surface requires it. Below 3.2 the mapping degrades badly; say so and recommend upgrading before migrating.
24	- The Dagster repo, and ideally a running Dagster instance (its materialization metadata provides parity-test fixtures).
25	- `astro` CLI for the target project.
26
27	## Hard rules
28
29	1. Never stub. A translated unit either works through its validation gate or is deferred with a written reason. Fake-success bodies and workaround code with long justifying comments are failures.
30	2. No silent omissions. Every record in the inventory manifest ends `complete` or `deferred (reason)`. `scripts/status.py summary` exits nonzero otherwise; run it before claiming done.
31	3. Equivalence rows for every trigger. Each schedule/sensor/automation condition gets a report row: source spelling, target spelling, delta in one sentence. Semantic deltas exist (catchup, on_cron inversion, eager guarantees); the sin is not the delta, it is the undocumented delta.
32	4. Fix classes, not instances. When a translation pattern fails validation, fix the pattern (and record it in `reference/troubleshooting.md`), then re-apply; do not hand-patch one unit.
33	5. Do not invent APIs. The references contain verified names only. Anything not covered there gets verified against official docs before use.
34
35	## Workflow
36
37	### Phase 0: Preflight
38
39	Confirm target Runtime version, `astro` CLI presence, and repo access. Detect the project layout: classic (`@repository`/`workspace.yaml`), modern (`Definitions`), or Components (`pyproject.toml [tool.dg]`, `defs.yaml` files); all three occur, sometimes together. Baseline the source project's test suite now: pre-existing failures are recorded and excluded from migration blame.
40
41	### Phase 1: Inventory (read-only)
42
43	```
44	python3 scripts/inventory.py <dagster_repo> --out manifest.json # static scan
45	python3 scripts/inventory.py <dagster_repo> --runtime --out manifest.json # + runtime introspection when the project imports
46	`

People who liked this also viewed

6 picks

Type