data-quality-and-contract-testing

Name: data-quality-and-contract-testing
Author: vaquarkhan

19stars4forks

How to install data-quality-and-contract-testing?

vaquarkhan/data-engineering-agent-skills/data-quality-and-contract-testing

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-quality-and-contract-testing

Installs into the current project.

Summary

Drives data implementation with contracts, assertions, and validation evidence. Use when adding or changing ingestion logic, transformations, schemas, or published data products.

Files · 1

View on GitHub

SKILL.md

1	# Data Quality And Contract Testing
2
3	## Overview
4
5	Data work is not complete when code runs. It is complete when source assumptions, output contracts, and quality checks prove the behavior is correct.
6
7	## When to Use
8
9	- new source ingestion
10	- schema changes
11	- transformation logic updates
12	- new or changed published tables
13	- bug fixes involving bad data or broken metrics
14
15	Do not use this only as a final cleanup step. It should guide implementation from the start.
16
17	## Workflow
18
19	1. Define the contract before implementation.
20	Capture:
21	- required fields
22	- key constraints
23	- expected types
24	- allowed null behavior
25	- freshness expectations
26	- reconciliation rules
27
28	2. Write the validation plan first.
29	Common validations:
30	- uniqueness
31	- non-null thresholds
32	- referential integrity
33	- accepted values
34	- row count deltas
35	- source-to-target totals
36
37	3. Reproduce data bugs with a failing check.
38	If an incident or defect exists, write the failing validation or test before changing the pipeline.
39
40	4. Implement the smallest change that satisfies the contract.
41
42	5. Run the relevant validations and capture evidence.
43	Evidence may include:
44	- test output
45	- query results
46	- sample reconciliation output
47	- dry-run logs
48
49	## Common Rationalizations
50
51	\| Rationalization \| Reality \|
52	\| --- \| --- \|
53	\| "The warehouse query looks right." \| Visual inspection does not scale and misses edge cases. \|
54	\| "We will add checks after the model stabilizes." \| Unchecked pipelines create low-trust data and harder incident response. \|
55	\| "A successful job means the data is valid." \| Jobs succeed while still producing incorrect or incomplete data. \|
56
57	## Red Flags
58
59	- no contract exists for a published dataset
60	- an incident fix ships without a failing reproduction check
61	- only happy-path sample data is validated
62	- freshness or completeness expectations are absent
63
64	## Verification
65
66	- [ ] Contracts are written before or alongside implementation
67	- [ ] Relevant checks exist for correctness, completeness, and freshness
68	- [ ] Defects are reproduced with a failing validation before the fix
69	- [ ] Evidence from validation is captured and reviewable

People who liked this also viewed

6 picks

Type

data-quality-and-contract-testing

19stars4forks

View on GitHub

How to install data-quality-and-contract-testing?

vaquarkhan/data-engineering-agent-skills/data-quality-and-contract-testing

$npx -y skills add vaquarkhan/data-engineering-agent-skills --skill data-quality-and-contract-testing

Installs into the current project.

Summary

Drives data implementation with contracts, assertions, and validation evidence. Use when adding or changing ingestion logic, transformations, schemas, or published data products.

Files · 1

View on GitHub

SKILL.md

1	# Data Quality And Contract Testing
2
3	## Overview
4
5	Data work is not complete when code runs. It is complete when source assumptions, output contracts, and quality checks prove the behavior is correct.
6
7	## When to Use
8
9	- new source ingestion
10	- schema changes
11	- transformation logic updates
12	- new or changed published tables
13	- bug fixes involving bad data or broken metrics
14
15	Do not use this only as a final cleanup step. It should guide implementation from the start.
16
17	## Workflow
18
19	1. Define the contract before implementation.
20	Capture:
21	- required fields
22	- key constraints
23	- expected types
24	- allowed null behavior
25	- freshness expectations
26	- reconciliation rules
27
28	2. Write the validation plan first.
29	Common validations:
30	- uniqueness
31	- non-null thresholds
32	- referential integrity
33	- accepted values
34	- row count deltas
35	- source-to-target totals
36
37	3. Reproduce data bugs with a failing check.
38	If an incident or defect exists, write the failing validation or test before changing the pipeline.
39
40	4. Implement the smallest change that satisfies the contract.
41
42	5. Run the relevant validations and capture evidence.
43	Evidence may include:
44	- test output
45	- query results
46	- sample reconciliation output
47	- dry-run logs
48
49	## Common Rationalizations
50
51	\| Rationalization \| Reality \|
52	\| --- \| --- \|
53	\| "The warehouse query looks right." \| Visual inspection does not scale and misses edge cases. \|
54	\| "We will add checks after the model stabilizes." \| Unchecked pipelines create low-trust data and harder incident response. \|
55	\| "A successful job means the data is valid." \| Jobs succeed while still producing incorrect or incomplete data. \|
56
57	## Red Flags
58
59	- no contract exists for a published dataset
60	- an incident fix ships without a failing reproduction check
61	- only happy-path sample data is validated
62	- freshness or completeness expectations are absent
63
64	## Verification
65
66	- [ ] Contracts are written before or alongside implementation
67	- [ ] Relevant checks exist for correctness, completeness, and freshness
68	- [ ] Defects are reproduced with a failing validation before the fix
69	- [ ] Evidence from validation is captured and reviewable

People who liked this also viewed

6 picks

Type