data-observability-and-sla-management

Name: data-observability-and-sla-management
Author: vaquarkhan

19stars4forks

How to install data-observability-and-sla-management?

vaquarkhan/data-engineering-agent-skills/data-observability-and-sla-management

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-observability-and-sla-management

Installs into the current project.

Summary

Guides agents through data observability and service-level management. Use when defining or improving freshness, completeness, anomaly detection, alerting, lag tracking, run metadata, and ownership for production data products.

Files · 1

View on GitHub

SKILL.md

1	# Data Observability And SLA Management
2
3	## Overview
4
5	Use this skill when the pipeline must be operated as a service, not just executed as code. It helps agents define what healthy looks like and how teams know when the system drifts away from that state.
6
7	## When to Use
8
9	- launching or hardening a production data product
10	- defining freshness or completeness SLAs
11	- improving alerting and anomaly detection
12	- adding operational metadata and health visibility
13	- reducing noisy or low-signal incident response
14	- designing resilience drills with explicit alert and recovery evidence
15
16	## Workflow
17
18	1. Define the service promises.
19	Include:
20	- freshness SLA
21	- completeness expectations
22	- acceptable latency
23	- owner and escalation path
24
25	2. Identify health signals.
26	Common signals:
27	- run success rate
28	- task duration drift
29	- volume anomalies
30	- schema drift
31	- consumer lag
32	- data freshness
33
34	3. Design alerts for actionability.
35	Alerts should route to someone who can act, with enough context to investigate quickly.
36
37	4. Capture run metadata and failure context.
38
39	5. Review alert quality.
40	Noisy alerts damage trust just as much as missing alerts.
41
42	6. Pair health signals with resilience drills when recovery behavior matters.
43	Load `references/data-resiliency-testing-patterns.md` when the team must prove restart, retry, backlog, or failover behavior under controlled failure.
44
45	## Common Rationalizations
46
47	\| Rationalization \| Reality \|
48	\| --- \| --- \|
49	\| "The scheduler already tells us if it fails." \| Task failure alone does not measure stale, partial, or bad data. \|
50	\| "More alerts are safer." \| Alert fatigue makes real incidents easier to miss. \|
51	\| "The business will tell us if something is wrong." \| That means the system failed before the team noticed. \|
52
53	## Red Flags
54
55	- no named owner or escalation path
56	- freshness is assumed but not measured
57	- alerts fire without run context or impact clues
58	- anomaly detection exists with no response playbook
59
60	## Verification
61
62	- [ ] SLAs and ownership are defined
63	- [ ] Health signals exist for freshness, completeness, and reliability
64	- [ ] Alerts are actionable and routed correctly
65	- [ ] Run metadata supports fast diagnosis

People who liked this also viewed

6 picks

Type

data-observability-and-sla-management

19stars4forks

View on GitHub

How to install data-observability-and-sla-management?

vaquarkhan/data-engineering-agent-skills/data-observability-and-sla-management

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-observability-and-sla-management

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Data Observability And SLA Management
2
3	## Overview
4
5	Use this skill when the pipeline must be operated as a service, not just executed as code. It helps agents define what healthy looks like and how teams know when the system drifts away from that state.
6
7	## When to Use
8
9	- launching or hardening a production data product
10	- defining freshness or completeness SLAs
11	- improving alerting and anomaly detection
12	- adding operational metadata and health visibility
13	- reducing noisy or low-signal incident response
14	- designing resilience drills with explicit alert and recovery evidence
15
16	## Workflow
17
18	1. Define the service promises.
19	Include:
20	- freshness SLA
21	- completeness expectations
22	- acceptable latency
23	- owner and escalation path
24
25	2. Identify health signals.
26	Common signals:
27	- run success rate
28	- task duration drift
29	- volume anomalies
30	- schema drift
31	- consumer lag
32	- data freshness
33
34	3. Design alerts for actionability.
35	Alerts should route to someone who can act, with enough context to investigate quickly.
36
37	4. Capture run metadata and failure context.
38
39	5. Review alert quality.
40	Noisy alerts damage trust just as much as missing alerts.
41
42	6. Pair health signals with resilience drills when recovery behavior matters.
43	Load `references/data-resiliency-testing-patterns.md` when the team must prove restart, retry, backlog, or failover behavior under controlled failure.
44
45	## Common Rationalizations
46
47	\| Rationalization \| Reality \|
48	\| --- \| --- \|
49	\| "The scheduler already tells us if it fails." \| Task failure alone does not measure stale, partial, or bad data. \|
50	\| "More alerts are safer." \| Alert fatigue makes real incidents easier to miss. \|
51	\| "The business will tell us if something is wrong." \| That means the system failed before the team noticed. \|
52
53	## Red Flags
54
55	- no named owner or escalation path
56	- freshness is assumed but not measured
57	- alerts fire without run context or impact clues
58	- anomaly detection exists with no response playbook
59
60	## Verification
61
62	- [ ] SLAs and ownership are defined
63	- [ ] Health signals exist for freshness, completeness, and reliability
64	- [ ] Alerts are actionable and routed correctly
65	- [ ] Run metadata supports fast diagnosis

People who liked this also viewed

6 picks

Type