authoring-go-sdk-tasks
installsforks
How to install authoring-go-sdk-tasks?
astronomer/agents/authoring-go-sdk-tasks
$npx -y skills add astronomer/agents --skill authoring-go-sdk-tasksWrites Airflow task logic in Go using the Airflow Go SDK. Use when the user wants to implement Airflow tasks in Go, asks about BundleProvider/RegisterDags, the bundlev1 Registry/Dag interfaces, registering Go tasks (AddTask/AddTaskWithName), dependency injection by para
SKILL.md
| 1 | # Authoring Go SDK Tasks |
| 2 | |
| 3 | The Airflow Go SDK implements the language-SDK model for Go: your DAG stays in Python, and each task is a compiled Go function registered inside a **bundle** (a single native executable). This skill covers the **Go-specific** native API. The shared model (the Python `@task.stub` pattern, ID matching, the XCom-as-JSON contract) lives in **authoring-language-sdk-tasks**; read that first if you are new to language SDKs. |
| 4 | |
| 5 | > **Experimental.** The Go SDK is under active development and not production-ready. Module path `github.com/apache/airflow/go-sdk` (Go 1.24+). APIs may change. |
| 6 | |
| 7 | > **Related skills:** **authoring-language-sdk-tasks** (shared Python stub + concepts), **deploying-go-sdk-bundles** (build, pack, and ship the bundle), **configuring-airflow-language-sdks** (route the queue to the Go coordinator). |
| 8 | |
| 9 | --- |
| 10 | |
| 11 | ## Recap: the Python side |
| 12 | |
| 13 | A Go task is paired with a Python stub that carries no logic; it declares the task, its queue, and the dependency graph. IDs must match the Go registration exactly, and `queue=` routes the task to the Go runtime. Full rules are in **authoring-language-sdk-tasks**; the minimal shape: |
| 14 | |
| 15 | ```python |
| 16 | from airflow.sdk import dag, task |
| 17 | |
| 18 | |
| 19 | @task.stub(queue="golang") |
| 20 | def extract(): ... |
| 21 | |
| 22 | |
| 23 | @task.stub(queue="golang") |
| 24 | def transform(): ... |
| 25 | |
| 26 | |
| 27 | @dag() |
| 28 | def simple_dag(): |
| 29 | extract() >> transform() |
| 30 | |
| 31 | |
| 32 | simple_dag() |
| 33 | ``` |
| 34 | |
| 35 | The `queue` value (`"golang"` here) is an arbitrary label that must match the queue routed to the Go coordinator (`queue_to_coordinator`). See **configuring-airflow-language-sdks**. |
| 36 | |
| 37 | --- |
| 38 | |
| 39 | ## The bundle entry point |
| 40 | |
| 41 | A bundle implements `bundlev1.BundleProvider`: report its version and register your DAGs and tasks. `main` is one line; `bundlev1server.Serve` wires the bundle to the Airflow runtime for you. |
| 42 | |
| 43 | ```go |
| 44 | package main |
| 45 | |
| 46 | import ( |
| 47 | "log" |
| 48 | |
| 49 | v1 "github.com/apache/airflow/go-sdk/bundle/bundlev1" |
| 50 | "github.com/apache/airflow/go-sdk/bundle/bundlev1/bundlev1server" |
| 51 | ) |
| 52 | |
| 53 | type myBundle struct{} |
| 54 | |
| 55 | var _ v1.BundleProvider = (*myBundle)(nil) |
| 56 | |
| 57 | func (m *myBundle) GetBundleVersion() v1.BundleInfo { |
| 58 | return v1.BundleInfo{Name: bundleName, Version: &bundleVersion} |
| 59 | } |
| 60 | |
| 61 | func (m *myBundle) RegisterDags(dagbag v1.Registry) error { |
| 62 | simpleDag := dagbag.AddDag("simple_dag") // dag_id must match the Python @dag name |
| 63 | simpleDag.AddTask(extract) // task_id is the function name; must match the stub |
| 64 | simpleDag.AddTaskWithName("transform", transform) // or set the task_id explicitly |
| 65 | return nil |
| 66 | } |
| 67 | |
| 68 | func main() { |
| 69 | if err := bundlev1server.Serve(&myBundle{}); err != nil { |
| 70 | log.Fatal(err) |
| 71 | } |
| 72 | } |
| 73 | ``` |
| 74 | |
| 75 | `AddTask(fn)` derives the `task_id` from the Go function's name; use `AddTaskWithName("<task_id>", fn)` when that name can't match the Python stub (an unexported, renamed, or reused function). `RegisterDags` is the single source of truth for task identity: the bundle's manifest (used by the packer and by the coordinator) is generated by running it, never hand-written. |
| 76 | |
| 77 | --- |
| 78 | |
| 79 | ## Task functions: dependency injection by parameter type |
| 80 | |
| 81 | A task is an ordinary Go function. The runtime inspects its signature and injects arguments **by type**; declare only what you need. |
| 82 | |
| 83 | | Parameter type | Injected value | |
| 84 | |----------------|----------------| |
| 85 | | `context.Context` | Task context for cancellation. Always available. | |
| 86 | | `sdk.TIRunContext` | Richer context (embeds `context.Context`) exposing `TaskInstance()` and `DagRun()`. See [Runtime context](#runtime-context). | |
| 87 | | `*slog.Logger` | Logger wired to the Airflow task log. | |
| 88 | | `sdk.Client` | Full Airflow model access: Variables, Connections, XComs. | |
| 89 | | `sdk.VariableClient` / `sdk.ConnectionClient` / `sdk.XComClient` | A narrower slice of `sdk.Client`. Prefer the narrowest you need; it documents intent and is trivial to fake in tests. | |
| 90 | |
| 91 | The optional return signature is `(result, error)`: a non-nil `result` is pushed as the task's `return_value` XCom; a non-nil `error` fails the task (which triggers the stub's retry policy). Returning only `error`, or nothing, is also valid. |
| 92 | |
| 93 | ```go |
| 94 | func extract(ctx sdk.TIRunContext, client sdk.Client, log *slog.Logger) (any, error) { |
| 95 | conn, err := client.GetConnection(ctx, "test_http") |
| 96 | if err != nil { |
| 97 | return nil, err |
| 98 | } |
| 99 | log.Info("connected", "host", conn.Host) |
| 100 | return map[string]any{" |
People who liked this also viewed
6 picksType
- hyperframes-cliUse the HyperFrames CLI development loop: init, add, catalog, capture, lint, check, snapshot, compare, grade-compare, preview, play, present, beats, keyframes, single or batch render, publish, cloud, SkillsJul 2026
- request-refactor-planSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026
- analyze-projectRigor Analyze / Rigor Audit read-only skill for deep learning research repositories. Use when the user wants to read and understand a repository, inspect model structure and training or inference entrSkillsMay 2026
- explore-codeRigor Improve implementation leaf skill for auditable candidate implementation in deep learning research repositories. Use when the researcher explicitly authorizes exploratory work on an isolated braSkillsMay 2026
- ubiquitous-languageSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026
- scaffold-exercisesSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026
