authoring-java-sdk-tasks

Name: authoring-java-sdk-tasks
Author: astronomer

10installs55forks

How to install authoring-java-sdk-tasks?

astronomer/agents/authoring-java-sdk-tasks

$npx -y skills add astronomer/agents --skill authoring-java-sdk-tasks

Installs into the current project.

Summary

Writes Airflow task logic in Java, Kotlin, or any JVM language using the Airflow Java SDK. Use when the user wants to implement Airflow tasks in Java/JVM, asks about @Builder.Dag/@Builder.Task/@Builder.XCom, the Task/BundleBuilder interfaces, reading connections/variabl

Files · 1

View on GitHub

SKILL.md

1	# Authoring Java SDK Tasks
2
3	The Airflow Java SDK implements the language-SDK model for the JVM: your DAG stays in Python, and each task instance runs in a short-lived JVM subprocess. This skill covers the Java-specific native API. The shared model — the Python `@task.stub` pattern, ID matching, and the XCom-as-JSON contract — lives in authoring-language-sdk-tasks; read that first if you're new to language SDKs.
4
5	> Experimental. The Java SDK is in preview. Artifact coordinates and APIs may change.
6
7	> Related skills: authoring-language-sdk-tasks (shared Python stub + concepts), configuring-airflow-language-sdks (route the queue to `JavaCoordinator`), deploying-java-sdk-bundles (compile and ship the JAR).
8
9	---
10
11	## Recap: the Python side
12
13	Java tasks are paired with Python stubs that carry no logic — they declare the task, queue, dependency graph, and retries. IDs must match the Java annotations exactly, and an upstream argument on a stub only declares the dependency (the value is fetched in Java). Full rules are in authoring-language-sdk-tasks; the minimal shape:
14
15	```python
16	from airflow.sdk import dag, task
17
18
19	@dag
20	def sales_pipeline(): # dag_id "sales_pipeline" -> @Builder.Dag(id="sales_pipeline")
21	@task.stub(queue="java")
22	def extract(): ... # task_id "extract" -> @Builder.Task(id="extract")
23
24	@task.stub(queue="java")
25	def transform(extracted): ...
26
27	transform(extract())
28
29
30	sales_pipeline()
31	```
32
33	---
34
35	## Java side: two APIs
36
37	Both APIs produce identical runtime behavior; pick by style, and you can mix them in one bundle.
38
39	### Annotation-based API (recommended)
40
41	Annotate a plain class; an annotation processor generates the wiring (`<ClassName>Builder`) at compile time.
42
43	```java
44	import static java.lang.System.Logger.Level.INFO;
45	import org.apache.airflow.sdk.*;
46
47	@Builder.Dag(id = "sales_pipeline") // must match the Python dag_id
48	public class SalesPipeline {
49	private static final System.Logger log = System.getLogger(SalesPipeline.class.getName());
50
51	@Builder.Task(id = "extract") // must match the Python @task.stub name
52	public long extract(Client client) {
53	var conn = client.getConnection("sales_db");
54	log.log(INFO, "connected to {0}", conn.host);
55	return 42L; // return value is pushed as the return_value XCom
56	}
57
58	@Builder.Task(id = "transform")
59	public long transform(
60	Client client,
61	@Builder.XCom(task = "extract") long recordCount) { // pulls extract's return_value
62	var threshold = (String) client.getVariable("transform_threshold");
63	return recordCount * 2;
64	}
65
66	@Builder.Task // id omitted -> the method name "load" is used
67	public void load(Context context, @Builder.XCom(task = "transform") long transformed) {
68	log.log(INFO, "attempt {0}, value {1}", context.ti.tryNumber, transformed);
69	}
70	}
71	```
72
73	Annotation reference:
74
75	\| Annotation \| Purpose \|
76	\|------------\|---------\|
77	\| `@Builder.Dag(id = "...")` \| Marks the class as a task container. `id` must match the Python `dag_id`; if omitted, the class name is used. Optional `to = "..."` renames the generated builder (default `<ClassName>Builder`). \|
78	\| `@Builder.Task(id = "...")` \| Marks a method as a task. `id` must match the Python `@task.stub` function name; if omitted, the method name is used. \|
79	\| `@Builder.XCom(task = "...", key = "...")` \| Injects an upstream task's XCom as a parameter. `task` defaults to the parameter name; `key` defaults to the producing task's `return_value`. The parameter type must be compatible with the stored JSON value. \|
80
81	A task method's return value is automatically pushed as that task's `return_value` XCom. A method may declare `throws Exception`; any uncaught exception fails the task instance (which triggers retries if the stub configured them).
82
83	### Interface-based API
84
85	Implement `Task` directly when you want full control over registration and XCom handling.
86
87	```java
88	import org.apache.airflow.sdk.*;
89
90	public class ExtractTask implements Task {
91	@Override
92	public void execute(Context context, Client client) throws Exception {
93	var conn = client.getConnection("sales_db");
94	// ... do work ...
95	client.setXCom(42L); // push return_value explicitly
96	}
97	}
98	```
99
100	Register tasks manually in a `Dag` a

People who liked this also viewed

6 picks

Type

authoring-java-sdk-tasks

10installs55forks

View on GitHub

How to install authoring-java-sdk-tasks?

astronomer/agents/authoring-java-sdk-tasks

$npx -y skills add astronomer/agents --skill authoring-java-sdk-tasks

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Authoring Java SDK Tasks
2
3	The Airflow Java SDK implements the language-SDK model for the JVM: your DAG stays in Python, and each task instance runs in a short-lived JVM subprocess. This skill covers the Java-specific native API. The shared model — the Python `@task.stub` pattern, ID matching, and the XCom-as-JSON contract — lives in authoring-language-sdk-tasks; read that first if you're new to language SDKs.
4
5	> Experimental. The Java SDK is in preview. Artifact coordinates and APIs may change.
6
7	> Related skills: authoring-language-sdk-tasks (shared Python stub + concepts), configuring-airflow-language-sdks (route the queue to `JavaCoordinator`), deploying-java-sdk-bundles (compile and ship the JAR).
8
9	---
10
11	## Recap: the Python side
12
13	Java tasks are paired with Python stubs that carry no logic — they declare the task, queue, dependency graph, and retries. IDs must match the Java annotations exactly, and an upstream argument on a stub only declares the dependency (the value is fetched in Java). Full rules are in authoring-language-sdk-tasks; the minimal shape:
14
15	```python
16	from airflow.sdk import dag, task
17
18
19	@dag
20	def sales_pipeline(): # dag_id "sales_pipeline" -> @Builder.Dag(id="sales_pipeline")
21	@task.stub(queue="java")
22	def extract(): ... # task_id "extract" -> @Builder.Task(id="extract")
23
24	@task.stub(queue="java")
25	def transform(extracted): ...
26
27	transform(extract())
28
29
30	sales_pipeline()
31	```
32
33	---
34
35	## Java side: two APIs
36
37	Both APIs produce identical runtime behavior; pick by style, and you can mix them in one bundle.
38
39	### Annotation-based API (recommended)
40
41	Annotate a plain class; an annotation processor generates the wiring (`<ClassName>Builder`) at compile time.
42
43	```java
44	import static java.lang.System.Logger.Level.INFO;
45	import org.apache.airflow.sdk.*;
46
47	@Builder.Dag(id = "sales_pipeline") // must match the Python dag_id
48	public class SalesPipeline {
49	private static final System.Logger log = System.getLogger(SalesPipeline.class.getName());
50
51	@Builder.Task(id = "extract") // must match the Python @task.stub name
52	public long extract(Client client) {
53	var conn = client.getConnection("sales_db");
54	log.log(INFO, "connected to {0}", conn.host);
55	return 42L; // return value is pushed as the return_value XCom
56	}
57
58	@Builder.Task(id = "transform")
59	public long transform(
60	Client client,
61	@Builder.XCom(task = "extract") long recordCount) { // pulls extract's return_value
62	var threshold = (String) client.getVariable("transform_threshold");
63	return recordCount * 2;
64	}
65
66	@Builder.Task // id omitted -> the method name "load" is used
67	public void load(Context context, @Builder.XCom(task = "transform") long transformed) {
68	log.log(INFO, "attempt {0}, value {1}", context.ti.tryNumber, transformed);
69	}
70	}
71	```
72
73	Annotation reference:
74
75	\| Annotation \| Purpose \|
76	\|------------\|---------\|
77	\| `@Builder.Dag(id = "...")` \| Marks the class as a task container. `id` must match the Python `dag_id`; if omitted, the class name is used. Optional `to = "..."` renames the generated builder (default `<ClassName>Builder`). \|
78	\| `@Builder.Task(id = "...")` \| Marks a method as a task. `id` must match the Python `@task.stub` function name; if omitted, the method name is used. \|
79	\| `@Builder.XCom(task = "...", key = "...")` \| Injects an upstream task's XCom as a parameter. `task` defaults to the parameter name; `key` defaults to the producing task's `return_value`. The parameter type must be compatible with the stored JSON value. \|
80
81	A task method's return value is automatically pushed as that task's `return_value` XCom. A method may declare `throws Exception`; any uncaught exception fails the task instance (which triggers retries if the stub configured them).
82
83	### Interface-based API
84
85	Implement `Task` directly when you want full control over registration and XCom handling.
86
87	```java
88	import org.apache.airflow.sdk.*;
89
90	public class ExtractTask implements Task {
91	@Override
92	public void execute(Context context, Client client) throws Exception {
93	var conn = client.getConnection("sales_db");
94	// ... do work ...
95	client.setXCom(42L); // push return_value explicitly
96	}
97	}
98	```
99
100	Register tasks manually in a `Dag` a

People who liked this also viewed

6 picks

Type