java-data-engineering-and-integration-services

Name: java-data-engineering-and-integration-services
Author: vaquarkhan

19stars4forks

How to install java-data-engineering-and-integration-services?

vaquarkhan/data-engineering-agent-skills/java-data-engineering-and-integration-services

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill java-data-engineering-and-integration-services

Installs into the current project.

Summary

Guides agents through Java-based data engineering services and processors. Use when building connectors, ingestion services, stream processors, metadata services, JVM batch tools, or operational integrations in Java.

Files · 1

View on GitHub

SKILL.md

1	# Java Data Engineering And Integration Services
2
3	## Overview
4
5	Use this skill when `Java` is the main implementation language for data-adjacent services or processing components. It helps agents design operationally safe JVM services for ingestion, metadata, contracts, stream handling, connectors, and control-plane style data tooling with deliberate resource, dependency, and concurrency management.
6
7	## When to Use
8
9	- building ingestion or connector services in `Java`
10	- implementing JVM-based stream processors or integration utilities
11	- exposing data-platform metadata, contract, or control services
12	- managing `Maven` or `Gradle` builds for data-related services
13	- debugging resource, thread, serialization, or connection-pool behavior in JVM services
14
15	Do not treat Java services as generic app code when they carry data-delivery, contract, or pipeline semantics.
16
17	## Workflow
18
19	1. Define the service role and operational boundary.
20	Clarify:
21	- request or event model
22	- upstream and downstream systems
23	- throughput and latency expectations
24	- delivery guarantees
25	- retry and failure behavior
26
27	2. Make contracts explicit.
28	Include:
29	- payload schemas
30	- versioning behavior
31	- idempotency rules
32	- error model
33	- compatibility with downstream consumers
34
35	3. Design resource and concurrency behavior deliberately.
36	Review:
37	- thread pools
38	- blocking versus async paths
39	- connection management
40	- backpressure
41	- graceful shutdown and restart behavior
42
43	4. Package and configure for operations.
44	Decide:
45	- `Maven` or `Gradle` conventions
46	- dependency version strategy
47	- environment configuration
48	- secrets handling
49	- observability and health signals
50
51	5. Validate service behavior under realistic load and failure conditions.
52	Require:
53	- contract checks
54	- retry and timeout tests
55	- connection and resource sanity
56	- release and rollback readiness
57
58	## Common Rationalizations
59
60	\| Rationalization \| Reality \|
61	\| --- \| --- \|
62	\| "The framework defaults are good enough." \| Defaults for thread pools, connection pools, and retries often fail under data-heavy or bursty workloads. \|
63	\| "It is just a connector wrapper." \| Connectors still define contracts, error handling, retries, and downstream correctness. \|
64	\| "Java is verbose but safe by default." \| JVM services still fail due to resource leaks, blocking calls, schema drift, and weak operational boundaries. \|
65
66	## Red Flags
67
68	- retry behavior can duplicate writes or downstream side effects
69	- thread, connection, or shutdown behavior is undocumented
70	- service config and secrets are embedded in code or build files
71	- payload compatibility or schema evolution is undefined
72	- observability is limited to basic process health with no data-path evidence
73
74	## Verification
75
76	- [ ] The Java service role and delivery semantics are explicit
77	- [ ] Contract, retry, and idempotency behavior are defined
78	- [ ] Resource, concurrency, and shutdown behavior are reviewed
79	- [ ] Build, config, and observability expectations are operationally real

People who liked this also viewed

6 picks

Type

java-data-engineering-and-integration-services

19stars4forks

View on GitHub

How to install java-data-engineering-and-integration-services?

vaquarkhan/data-engineering-agent-skills/java-data-engineering-and-integration-services

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill java-data-engineering-and-integration-services

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Java Data Engineering And Integration Services
2
3	## Overview
4
5	Use this skill when `Java` is the main implementation language for data-adjacent services or processing components. It helps agents design operationally safe JVM services for ingestion, metadata, contracts, stream handling, connectors, and control-plane style data tooling with deliberate resource, dependency, and concurrency management.
6
7	## When to Use
8
9	- building ingestion or connector services in `Java`
10	- implementing JVM-based stream processors or integration utilities
11	- exposing data-platform metadata, contract, or control services
12	- managing `Maven` or `Gradle` builds for data-related services
13	- debugging resource, thread, serialization, or connection-pool behavior in JVM services
14
15	Do not treat Java services as generic app code when they carry data-delivery, contract, or pipeline semantics.
16
17	## Workflow
18
19	1. Define the service role and operational boundary.
20	Clarify:
21	- request or event model
22	- upstream and downstream systems
23	- throughput and latency expectations
24	- delivery guarantees
25	- retry and failure behavior
26
27	2. Make contracts explicit.
28	Include:
29	- payload schemas
30	- versioning behavior
31	- idempotency rules
32	- error model
33	- compatibility with downstream consumers
34
35	3. Design resource and concurrency behavior deliberately.
36	Review:
37	- thread pools
38	- blocking versus async paths
39	- connection management
40	- backpressure
41	- graceful shutdown and restart behavior
42
43	4. Package and configure for operations.
44	Decide:
45	- `Maven` or `Gradle` conventions
46	- dependency version strategy
47	- environment configuration
48	- secrets handling
49	- observability and health signals
50
51	5. Validate service behavior under realistic load and failure conditions.
52	Require:
53	- contract checks
54	- retry and timeout tests
55	- connection and resource sanity
56	- release and rollback readiness
57
58	## Common Rationalizations
59
60	\| Rationalization \| Reality \|
61	\| --- \| --- \|
62	\| "The framework defaults are good enough." \| Defaults for thread pools, connection pools, and retries often fail under data-heavy or bursty workloads. \|
63	\| "It is just a connector wrapper." \| Connectors still define contracts, error handling, retries, and downstream correctness. \|
64	\| "Java is verbose but safe by default." \| JVM services still fail due to resource leaks, blocking calls, schema drift, and weak operational boundaries. \|
65
66	## Red Flags
67
68	- retry behavior can duplicate writes or downstream side effects
69	- thread, connection, or shutdown behavior is undocumented
70	- service config and secrets are embedded in code or build files
71	- payload compatibility or schema evolution is undefined
72	- observability is limited to basic process health with no data-path evidence
73
74	## Verification
75
76	- [ ] The Java service role and delivery semantics are explicit
77	- [ ] Contract, retry, and idempotency behavior are defined
78	- [ ] Resource, concurrency, and shutdown behavior are reviewed
79	- [ ] Build, config, and observability expectations are operationally real

People who liked this also viewed

6 picks

Type