data-quality-checker

Name: data-quality-checker
Author: mphinance

15stars3forks

How to install data-quality-checker?

mphinance/alpha-skills/data-quality-checker

$npx -y skills add mphinance/alpha-skills --skill data-quality-checker

Installs into the current project.

Summary

Validate data quality in market analysis documents and blog articles before publication. Use when checking for price scale inconsistencies (ETF vs futures), instrument notation errors, date/day-of-week mismatches, allocation total errors, and unit mismatches. Supports English and

Files · 1

View on GitHub

SKILL.md

1	## Overview
2
3	Detect common data quality issues in market analysis documents before
4	publication. The checker validates five categories: price scale consistency,
5	instrument notation, date/weekday accuracy, allocation totals, and unit usage.
6	All findings are advisory -- they flag potential issues for human review rather
7	than blocking publication.
8
9	## When to Use
10
11	- Before publishing a weekly strategy blog or market analysis report
12	- After generating automated market summaries
13	- When reviewing translated documents (English/Japanese) for data accuracy
14	- When combining data from multiple sources (FRED, FMP, FINVIZ) into one report
15	- As a pre-flight check for any document containing financial data
16
17	## Prerequisites
18
19	- Python 3.9+
20	- No external API keys required
21	- No third-party Python packages required (uses only standard library)
22
23	## Workflow
24
25	### Step 1: Receive Input Document
26
27	Accept the target markdown file path and optional parameters:
28	- `--file`: Path to the markdown document to validate (required)
29	- `--checks`: Comma-separated list of checks to run (optional; default: all)
30	- `--as-of`: Reference date for year inference in YYYY-MM-DD format (optional)
31	- `--output-dir`: Directory for report output (optional; default: `reports/`)
32
33	### Step 2: Execute Validation Script
34
35	Run the data quality checker script:
36
37	```bash
38	python3 skills/data-quality-checker/scripts/check_data_quality.py \
39	--file path/to/document.md \
40	--output-dir reports/
41	```
42
43	To run specific checks only:
44
45	```bash
46	python3 skills/data-quality-checker/scripts/check_data_quality.py \
47	--file path/to/document.md \
48	--checks price_scale,dates,allocations
49	```
50
51	To provide a reference date for year inference (useful for documents without
52	explicit year in dates):
53
54	```bash
55	python3 skills/data-quality-checker/scripts/check_data_quality.py \
56	--file path/to/document.md \
57	--as-of 2026-02-28
58	```
59
60	### Step 3: Load Reference Standards
61
62	Read the relevant reference documents to contextualize findings:
63
64	- `references/instrument_notation_standard.md` -- Standard ticker notation,
65	digit-count hints, and naming conventions for each instrument class
66	- `references/common_data_errors.md` -- Catalog of frequently observed errors
67	including FRED data delays, ETF/futures scale confusion, holiday oversights,
68	allocation total pitfalls, and unit confusion patterns
69
70	Use these references to explain findings and suggest corrections.
71
72	### Step 4: Review Findings
73
74	Examine each finding in the output:
75
76	- ERROR -- High confidence issues (e.g., date-weekday mismatches verified
77	by calendar computation). Strongly recommend correction.
78	- WARNING -- Likely issues that need human judgment (e.g., price scale
79	anomalies, notation inconsistencies, allocation sums off by more than 0.5%).
80	- INFO -- Informational notes (e.g., mixed bp/% usage that may be
81	intentional).
82
83	### Step 5: Generate Quality Report
84
85	The script produces two output files:
86
87	1. JSON report (`data_quality_YYYY-MM-DD_HHMMSS.json`): Machine-readable
88	list of findings with severity, category, message, line number, and context.
89	2. Markdown report (`data_quality_YYYY-MM-DD_HHMMSS.md`): Human-readable
90	report grouped by severity level.
91
92	Present the findings to the user with explanations referencing the knowledge
93	base. Suggest specific corrections for each issue.
94
95	## Output Format
96
97	### JSON Finding Structure
98
99	```json
100	{
101	"severity": "WARNING",
102	"category": "price_scale",
103	"message": "GLD: $2,800 has 4 digits (expected 2-3 digits)",
104	"line_number": 5,
105	"context": "GLD: $2,800"
106	}
107	```
108
109	### Markdown Report Structure
110
111	```markdown
112	# Data Quality Report
113	Source: path/to/document.md
114	Generated: 2026-02-28 14:30:00
115	Total findings: 3
116
117	## ERROR (1)
118	- [dates] (line 12): Date-weekday mismatch: January 1, 2026 (Monday) -- actual weekday is Thursday
119
120	## WARNING (2)
121	- [price_scale] (line 5): GLD: $2,800 has 4 digits (expected 2-3 digits)
122	> `GLD: $2,800`
123	- [allocations]: Allocation total: 110.0% (expected ~100%)
124	```
125
126	## Resources
127
128	- `scripts/check_data_quality.py` -- Main validation script
129	- `references/instrument_notation_standard.md` -- Notation and price scale reference
130	- `references/common_data_errors.md` -- Common error patterns and prevention
131
132	## Key Principles
133
134	1. Advisory mode: All findings are warnings for human review. The script
135	always exits with code 0 on successful execution, even when findings are
136	present. Exit code 1 is reserved for script failures (file not found, parse
137	errors).
138
139	2. Section-aware allocation checking: Only percentages withi

People who liked this also viewed

6 picks

Type

data-quality-checker

15stars3forks

View on GitHub

How to install data-quality-checker?

mphinance/alpha-skills/data-quality-checker

$npx -y skills add mphinance/alpha-skills --skill data-quality-checker

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	## Overview
2
3	Detect common data quality issues in market analysis documents before
4	publication. The checker validates five categories: price scale consistency,
5	instrument notation, date/weekday accuracy, allocation totals, and unit usage.
6	All findings are advisory -- they flag potential issues for human review rather
7	than blocking publication.
8
9	## When to Use
10
11	- Before publishing a weekly strategy blog or market analysis report
12	- After generating automated market summaries
13	- When reviewing translated documents (English/Japanese) for data accuracy
14	- When combining data from multiple sources (FRED, FMP, FINVIZ) into one report
15	- As a pre-flight check for any document containing financial data
16
17	## Prerequisites
18
19	- Python 3.9+
20	- No external API keys required
21	- No third-party Python packages required (uses only standard library)
22
23	## Workflow
24
25	### Step 1: Receive Input Document
26
27	Accept the target markdown file path and optional parameters:
28	- `--file`: Path to the markdown document to validate (required)
29	- `--checks`: Comma-separated list of checks to run (optional; default: all)
30	- `--as-of`: Reference date for year inference in YYYY-MM-DD format (optional)
31	- `--output-dir`: Directory for report output (optional; default: `reports/`)
32
33	### Step 2: Execute Validation Script
34
35	Run the data quality checker script:
36
37	```bash
38	python3 skills/data-quality-checker/scripts/check_data_quality.py \
39	--file path/to/document.md \
40	--output-dir reports/
41	```
42
43	To run specific checks only:
44
45	```bash
46	python3 skills/data-quality-checker/scripts/check_data_quality.py \
47	--file path/to/document.md \
48	--checks price_scale,dates,allocations
49	```
50
51	To provide a reference date for year inference (useful for documents without
52	explicit year in dates):
53
54	```bash
55	python3 skills/data-quality-checker/scripts/check_data_quality.py \
56	--file path/to/document.md \
57	--as-of 2026-02-28
58	```
59
60	### Step 3: Load Reference Standards
61
62	Read the relevant reference documents to contextualize findings:
63
64	- `references/instrument_notation_standard.md` -- Standard ticker notation,
65	digit-count hints, and naming conventions for each instrument class
66	- `references/common_data_errors.md` -- Catalog of frequently observed errors
67	including FRED data delays, ETF/futures scale confusion, holiday oversights,
68	allocation total pitfalls, and unit confusion patterns
69
70	Use these references to explain findings and suggest corrections.
71
72	### Step 4: Review Findings
73
74	Examine each finding in the output:
75
76	- ERROR -- High confidence issues (e.g., date-weekday mismatches verified
77	by calendar computation). Strongly recommend correction.
78	- WARNING -- Likely issues that need human judgment (e.g., price scale
79	anomalies, notation inconsistencies, allocation sums off by more than 0.5%).
80	- INFO -- Informational notes (e.g., mixed bp/% usage that may be
81	intentional).
82
83	### Step 5: Generate Quality Report
84
85	The script produces two output files:
86
87	1. JSON report (`data_quality_YYYY-MM-DD_HHMMSS.json`): Machine-readable
88	list of findings with severity, category, message, line number, and context.
89	2. Markdown report (`data_quality_YYYY-MM-DD_HHMMSS.md`): Human-readable
90	report grouped by severity level.
91
92	Present the findings to the user with explanations referencing the knowledge
93	base. Suggest specific corrections for each issue.
94
95	## Output Format
96
97	### JSON Finding Structure
98
99	```json
100	{
101	"severity": "WARNING",
102	"category": "price_scale",
103	"message": "GLD: $2,800 has 4 digits (expected 2-3 digits)",
104	"line_number": 5,
105	"context": "GLD: $2,800"
106	}
107	```
108
109	### Markdown Report Structure
110
111	```markdown
112	# Data Quality Report
113	Source: path/to/document.md
114	Generated: 2026-02-28 14:30:00
115	Total findings: 3
116
117	## ERROR (1)
118	- [dates] (line 12): Date-weekday mismatch: January 1, 2026 (Monday) -- actual weekday is Thursday
119
120	## WARNING (2)
121	- [price_scale] (line 5): GLD: $2,800 has 4 digits (expected 2-3 digits)
122	> `GLD: $2,800`
123	- [allocations]: Allocation total: 110.0% (expected ~100%)
124	```
125
126	## Resources
127
128	- `scripts/check_data_quality.py` -- Main validation script
129	- `references/instrument_notation_standard.md` -- Notation and price scale reference
130	- `references/common_data_errors.md` -- Common error patterns and prevention
131
132	## Key Principles
133
134	1. Advisory mode: All findings are warnings for human review. The script
135	always exits with code 0 on successful execution, even when findings are
136	present. Exit code 1 is reserved for script failures (file not found, parse
137	errors).
138
139	2. Section-aware allocation checking: Only percentages withi

People who liked this also viewed

6 picks

Type