deploying-go-sdk-bundles

Name: deploying-go-sdk-bundles
Author: astronomer

26installs55forks

How to install deploying-go-sdk-bundles?

astronomer/agents/deploying-go-sdk-bundles

$npx -y skills add astronomer/agents --skill deploying-go-sdk-bundles

Installs into the current project.

Summary

Builds, packs, and deploys compiled Airflow Go SDK bundles so the ExecutableCoordinator can run them. Use when the user wants to compile a Go task bundle, asks about go build, go tool airflow-go-pack, the AFBNDL01 self-contained executable bundle, packing or inspecting a bund

Files · 1

View on GitHub

SKILL.md

1	# Deploying Go SDK Bundles
2
3	A Go SDK deployment has one artifact: a bundle, a single self-contained native executable that also carries its embedded source and a manifest (the AFBNDL01 format, "the executable is the bundle"). You build and pack it with `go`, place it where Airflow's `ExecutableCoordinator` scans, and the Python task runner forks it once per task instance. This skill is platform-neutral: it shows the build, the coordinator wiring, then how to get the bundle onto a worker.
4
5	> Experimental. The Go SDK is under active development and not production-ready. Everything resolves against the single module `github.com/apache/airflow/go-sdk` (Go 1.24+).
6
7	> Order of operations: write the tasks (authoring-go-sdk-tasks) -> build and pack the bundle (this skill) -> place it under `executables_root` and configure the coordinator -> deploy the matching Python stub DAG.
8
9	---
10
11	## Build and pack the bundle
12
13	The coordinator only recognizes a packed bundle: it scans for the AFBNDL01 trailer and silently skips any file that lacks it, so a plain `go build` binary is not deployable on its own. Use the packer, shipped as a Go 1.24 `tool` directive in `go.mod` (no global install, version pinned per project):
14
15	```bash
16	go tool airflow-go-pack ./example/bundle # build + pack in one step
17	go tool airflow-go-pack --goos linux --goarch amd64 ./example/bundle -- -trimpath # cross-compile; flags after -- pass to `go build`
18	go tool airflow-go-pack --executable ./bin/sample-dag-bundle --source main.go --airflow-metadata <airflow-metadata.yaml> # pack an existing binary
19	go tool airflow-go-pack inspect ./bin/sample-dag-bundle # inspect a packed bundle
20	```
21
22	The packer builds the binary, execs it with `--airflow-metadata` to capture the manifest from `RegisterDags`, then appends source + manifest + a 64-byte trailer. The result is one runnable file.
23
24	- Build for the worker's OS/arch. The bundle is a native executable and is not portable; cross-compile with `--goos`/`--goarch`. A mismatched binary fails on the worker with `exec format error`.
25	- Re-pack after any change to the binary. Re-stripping, re-signing, or swapping in a debug build invalidates the trailer's `binary_sha256`, and the bundle is then rejected.
26
27	---
28
29	## Wire up the coordinator
30
31	Python's `ExecutableCoordinator` scans `executables_root`, matches the incoming `dag_id` against each bundle's embedded manifest, verifies its integrity hash, then forks the bundle. No Go process runs on the host.
32
33	1. Place the packed executable under a scanned directory:
34
35	```bash
36	cp ./bundle /opt/airflow/executable-bundles/ # identified by the AFBNDL01 trailer, not by filename
37	```
38
39	2. Register `ExecutableCoordinator` and route the queue to it (see configuring-airflow-language-sdks):
40
41	```ini
42	[sdk]
43	coordinators = {"go": {"classpath": "airflow.sdk.coordinators.executable.ExecutableCoordinator", "kwargs": {"executables_root": ["/opt/airflow/executable-bundles"]}}}
44	queue_to_coordinator = {"golang": "go"}
45	```
46
47	3. Deploy the matching Python stub DAG; its `queue=` must equal the `queue_to_coordinator` key (`golang` here), and its `dag_id`/`task_id`s must match what the bundle registered.
48
49	---
50
51	## Deployment paths
52
53	The SDK runs on any Airflow with the Task SDK; Astronomer tooling is not required.
54
55	### Docker / Kubernetes
56
57	Cross-compile the bundle for the image's platform and bake it in. No Go runtime or worker process is needed in the image; the Python task runner forks the bundle.
58
59	```dockerfile
60	FROM apache/airflow:3.3.0 # the language SDKs target Airflow 3.3+
61	COPY ./executable-bundles/ /opt/airflow/executable-bundles/
62	# set AIRFLOW__SDK__COORDINATORS and AIRFLOW__SDK__QUEUE_TO_COORDINATOR as env vars
63	```
64
65	On the Helm chart, bake the bundle into a custom image as above or mount it via a shared volume, and set the `[sdk]` config through environment variables on the worker/scheduler. See deploying-airflow for the broader Docker Compose and Helm workflow.
66
67	> The `apache/airflow:3.3.0` tag above is illustrative: the language SDKs need Airflow 3.3 or newer. Pin whatever current 3.x you actually run rather than copying this tag from memory; read the base image's current tags or docs.
68
69	### Astro (one option, not required)
70
71	1. Build/pack the bundl

astronomerPublisher · 51 listings

Preview

astronomer/agents

# Deploying Go SDK Bundles

A Go SDK deployment has one artifact: a **bundle**, a single self-contained native executable that also carries its embedded source and a manifest (the AFBNDL01

> **Experimental.** The Go SDK is under active development and not production-ready. Everything resolves against the single module `github.com/apache/airflow/go

> **Order of operations:** write the tasks (**authoring-go-sdk-tasks**) -> build and pack the bundle (this skill) -> place it under `executables_root` and confi

Repoastronomer/agents

LicenseApache-2.0

TypeSkills

CategoryDevTools

UpdatedJul 2026

Repo stars411

First seenJul 22, 2026

People who liked this also viewed

6 picks

Type

deploying-go-sdk-bundles

26installs55forks

View on GitHub

How to install deploying-go-sdk-bundles?

astronomer/agents/deploying-go-sdk-bundles

$npx -y skills add astronomer/agents --skill deploying-go-sdk-bundles

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Deploying Go SDK Bundles
2
3	A Go SDK deployment has one artifact: a bundle, a single self-contained native executable that also carries its embedded source and a manifest (the AFBNDL01 format, "the executable is the bundle"). You build and pack it with `go`, place it where Airflow's `ExecutableCoordinator` scans, and the Python task runner forks it once per task instance. This skill is platform-neutral: it shows the build, the coordinator wiring, then how to get the bundle onto a worker.
4
5	> Experimental. The Go SDK is under active development and not production-ready. Everything resolves against the single module `github.com/apache/airflow/go-sdk` (Go 1.24+).
6
7	> Order of operations: write the tasks (authoring-go-sdk-tasks) -> build and pack the bundle (this skill) -> place it under `executables_root` and configure the coordinator -> deploy the matching Python stub DAG.
8
9	---
10
11	## Build and pack the bundle
12
13	The coordinator only recognizes a packed bundle: it scans for the AFBNDL01 trailer and silently skips any file that lacks it, so a plain `go build` binary is not deployable on its own. Use the packer, shipped as a Go 1.24 `tool` directive in `go.mod` (no global install, version pinned per project):
14
15	```bash
16	go tool airflow-go-pack ./example/bundle # build + pack in one step
17	go tool airflow-go-pack --goos linux --goarch amd64 ./example/bundle -- -trimpath # cross-compile; flags after -- pass to `go build`
18	go tool airflow-go-pack --executable ./bin/sample-dag-bundle --source main.go --airflow-metadata <airflow-metadata.yaml> # pack an existing binary
19	go tool airflow-go-pack inspect ./bin/sample-dag-bundle # inspect a packed bundle
20	```
21
22	The packer builds the binary, execs it with `--airflow-metadata` to capture the manifest from `RegisterDags`, then appends source + manifest + a 64-byte trailer. The result is one runnable file.
23
24	- Build for the worker's OS/arch. The bundle is a native executable and is not portable; cross-compile with `--goos`/`--goarch`. A mismatched binary fails on the worker with `exec format error`.
25	- Re-pack after any change to the binary. Re-stripping, re-signing, or swapping in a debug build invalidates the trailer's `binary_sha256`, and the bundle is then rejected.
26
27	---
28
29	## Wire up the coordinator
30
31	Python's `ExecutableCoordinator` scans `executables_root`, matches the incoming `dag_id` against each bundle's embedded manifest, verifies its integrity hash, then forks the bundle. No Go process runs on the host.
32
33	1. Place the packed executable under a scanned directory:
34
35	```bash
36	cp ./bundle /opt/airflow/executable-bundles/ # identified by the AFBNDL01 trailer, not by filename
37	```
38
39	2. Register `ExecutableCoordinator` and route the queue to it (see configuring-airflow-language-sdks):
40
41	```ini
42	[sdk]
43	coordinators = {"go": {"classpath": "airflow.sdk.coordinators.executable.ExecutableCoordinator", "kwargs": {"executables_root": ["/opt/airflow/executable-bundles"]}}}
44	queue_to_coordinator = {"golang": "go"}
45	```
46
47	3. Deploy the matching Python stub DAG; its `queue=` must equal the `queue_to_coordinator` key (`golang` here), and its `dag_id`/`task_id`s must match what the bundle registered.
48
49	---
50
51	## Deployment paths
52
53	The SDK runs on any Airflow with the Task SDK; Astronomer tooling is not required.
54
55	### Docker / Kubernetes
56
57	Cross-compile the bundle for the image's platform and bake it in. No Go runtime or worker process is needed in the image; the Python task runner forks the bundle.
58
59	```dockerfile
60	FROM apache/airflow:3.3.0 # the language SDKs target Airflow 3.3+
61	COPY ./executable-bundles/ /opt/airflow/executable-bundles/
62	# set AIRFLOW__SDK__COORDINATORS and AIRFLOW__SDK__QUEUE_TO_COORDINATOR as env vars
63	```
64
65	On the Helm chart, bake the bundle into a custom image as above or mount it via a shared volume, and set the `[sdk]` config through environment variables on the worker/scheduler. See deploying-airflow for the broader Docker Compose and Helm workflow.
66
67	> The `apache/airflow:3.3.0` tag above is illustrative: the language SDKs need Airflow 3.3 or newer. Pin whatever current 3.x you actually run rather than copying this tag from memory; read the base image's current tags or docs.
68
69	### Astro (one option, not required)
70
71	1. Build/pack the bundl

astronomerPublisher · 51 listings

Preview

astronomer/agents

# Deploying Go SDK Bundles

A Go SDK deployment has one artifact: a **bundle**, a single self-contained native executable that also carries its embedded source and a manifest (the AFBNDL01

> **Experimental.** The Go SDK is under active development and not production-ready. Everything resolves against the single module `github.com/apache/airflow/go

> **Order of operations:** write the tasks (**authoring-go-sdk-tasks**) -> build and pack the bundle (this skill) -> place it under `executables_root` and confi

Repoastronomer/agents

LicenseApache-2.0

TypeSkills

CategoryDevTools

UpdatedJul 2026

Repo stars411

First seenJul 22, 2026

People who liked this also viewed

6 picks

Type