data-specification

Name: data-specification
Author: vaquarkhan

19stars4forks

How to install data-specification?

vaquarkhan/data-engineering-agent-skills/data-specification

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-specification

Installs into the current project.

Summary

Creates structured specifications for data products and pipeline changes. Use when starting a new pipeline, model, ingestion flow, or any significant change with unclear requirements.

Files · 1

View on GitHub

SKILL.md

1	# Data Specification
2
3	## Overview
4
5	Write the data specification before writing pipeline code. The spec should define business intent, source and destination expectations, quality rules, and success criteria so the agent is not forced to guess.
6
7	## When to Use
8
9	- new ingestion or transformation projects
10	- schema or contract changes
11	- major changes to data products, marts, or semantic models
12	- requests that sound simple but leave operational details unclear
13
14	Do not use this for trivial spelling fixes or non-behavioral documentation edits.
15
16	## Workflow
17
18	1. State assumptions up front.
19	Include:
20	- business objective
21	- source systems
22	- destination systems
23	- data grain
24	- update cadence
25	- retention expectations
26	- security or privacy constraints
27
28	2. Write the specification around required sections.
29	- Objective
30	- Business outcomes
31	- Source systems and contracts
32	- Destination tables, files, or streams
33	- Freshness and SLA expectations
34	- Data quality rules
35	- Security and access boundaries
36	- Backfill and replay expectations
37	- Success criteria
38	- Open questions
39
40	3. Resolve ambiguity before planning.
41	If the spec cannot answer frequency, grain, keys, slowly changing behavior, or null handling, pause and ask.
42
43	4. Save the spec in version control.
44	A data change without a written spec becomes tribal knowledge.
45
46	## Common Rationalizations
47
48	\| Rationalization \| Reality \|
49	\| --- \| --- \|
50	\| "We just need the table built quickly." \| The wrong grain or contract creates expensive downstream rework. \|
51	\| "We can infer the business metric later." \| That usually creates multiple conflicting definitions of the same metric. \|
52	\| "The destination schema is enough." \| Schedules, freshness, backfills, and access rules matter just as much as columns. \|
53
54	## Red Flags
55
56	- no business owner is named
57	- source-of-truth systems are unclear
58	- success is defined as "pipeline runs"
59	- backfill behavior is omitted
60	- quality rules are implied instead of written
61
62	## Verification
63
64	- [ ] The spec states the business outcome and expected users
65	- [ ] Source, destination, grain, cadence, and ownership are explicit
66	- [ ] Quality, security, and replay expectations are defined
67	- [ ] Open questions are listed instead of guessed
68	- [ ] The spec is saved in the repository

People who liked this also viewed

6 picks

Type

data-specification

19stars4forks

View on GitHub

How to install data-specification?

vaquarkhan/data-engineering-agent-skills/data-specification

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-specification

Installs into the current project.

Summary

Creates structured specifications for data products and pipeline changes. Use when starting a new pipeline, model, ingestion flow, or any significant change with unclear requirements.

Files · 1

View on GitHub

SKILL.md

1	# Data Specification
2
3	## Overview
4
5	Write the data specification before writing pipeline code. The spec should define business intent, source and destination expectations, quality rules, and success criteria so the agent is not forced to guess.
6
7	## When to Use
8
9	- new ingestion or transformation projects
10	- schema or contract changes
11	- major changes to data products, marts, or semantic models
12	- requests that sound simple but leave operational details unclear
13
14	Do not use this for trivial spelling fixes or non-behavioral documentation edits.
15
16	## Workflow
17
18	1. State assumptions up front.
19	Include:
20	- business objective
21	- source systems
22	- destination systems
23	- data grain
24	- update cadence
25	- retention expectations
26	- security or privacy constraints
27
28	2. Write the specification around required sections.
29	- Objective
30	- Business outcomes
31	- Source systems and contracts
32	- Destination tables, files, or streams
33	- Freshness and SLA expectations
34	- Data quality rules
35	- Security and access boundaries
36	- Backfill and replay expectations
37	- Success criteria
38	- Open questions
39
40	3. Resolve ambiguity before planning.
41	If the spec cannot answer frequency, grain, keys, slowly changing behavior, or null handling, pause and ask.
42
43	4. Save the spec in version control.
44	A data change without a written spec becomes tribal knowledge.
45
46	## Common Rationalizations
47
48	\| Rationalization \| Reality \|
49	\| --- \| --- \|
50	\| "We just need the table built quickly." \| The wrong grain or contract creates expensive downstream rework. \|
51	\| "We can infer the business metric later." \| That usually creates multiple conflicting definitions of the same metric. \|
52	\| "The destination schema is enough." \| Schedules, freshness, backfills, and access rules matter just as much as columns. \|
53
54	## Red Flags
55
56	- no business owner is named
57	- source-of-truth systems are unclear
58	- success is defined as "pipeline runs"
59	- backfill behavior is omitted
60	- quality rules are implied instead of written
61
62	## Verification
63
64	- [ ] The spec states the business outcome and expected users
65	- [ ] Source, destination, grain, cadence, and ownership are explicit
66	- [ ] Quality, security, and replay expectations are defined
67	- [ ] Open questions are listed instead of guessed
68	- [ ] The spec is saved in the repository

People who liked this also viewed

6 picks

Type