analyst

Name: analyst
Author: QinghongLin

146stars19forks

How to install analyst?

qinghonglin/data2story-skill/analyst

$npx -y skills add QinghongLin/data2story-skill --skill analyst

Installs into the current project.

Summary

Exhaustively profile a dataset and list ALL possible analyses — distributions, correlations, rankings, trends, group comparisons, anomalies. Reads detective.json for context. Runs after the Detective/Scout, before the Editor. Outputs analyst.json with ana_xx IDs and chart-ready d

Files · 1

View on GitHub

SKILL.md

1	# Analyst
2
3	Your job is completeness, not curation. List every analysis this dataset can support, grounded in the context the Detective found. You are not deciding what story to tell — that is the Editor's job. You are cataloguing what the data contains.
4
5	## Setup
6
7	- `DATA_DIR` = first argument
8	- `PROJECT_DIR` = second argument
9	- Read `PROJECT_DIR/detective.json` before starting — it tells you what matters in this domain
10	- Also read `PROJECT_DIR/scout.json` if present — surface any `live_status[]` entries as display-only context (cite the dated source; never feed live status to a forecasting/training model)
11	- Outputs: `PROJECT_DIR/code/*.py` (analysis scripts), `PROJECT_DIR/analyst.json`
12
13	## Steps
14
15	### 1. Dataset Profile
16
17	Run code to compute:
18	- File(s), format, row count, column count
19	- What one row represents
20	- Time range, geographic scope
21	- Missing value counts per column
22	- Cardinality of categorical columns
23
24	### 2. Field Inventory
25
26	For every column:
27	- Name, inferred meaning, data type
28	- Sample values
29	- Noteworthy distributions or quirks
30
31	### 3. All Possible Analyses
32
33	Run actual code (Python/Bash) for every applicable category below.
34	Record the actual numbers — not descriptions of what could be computed.
35
36	Distributions — value counts for every categorical field; histogram buckets for every numeric field; null/missing rates.
37
38	Rankings — top and bottom N for every meaningful dimension; concentration (what % of outcomes does the top 10% account for?).
39
40	Group Comparisons — every categorical field as a grouping variable against every numeric/outcome field; note effect size, not just direction.
41
42	Correlations & Relationships — pairwise relationships between numeric fields; categorical interactions (e.g. A × B → outcome).
43
44	Trends & Sequences — time-based patterns if a date/order field exists; first vs. last, early vs. late.
45
46	Anomalies — values more than 2 SD from mean; unexpected zeros, near-perfect concentrations, impossible combinations.
47
48	Experiment-specific — if this is a study/survey: check for order effects, experimenter effects, condition imbalances.
49
50	Context-informed — use `detective.json` items to run any comparisons that have external benchmarks; flag where the data confirms, contradicts, or extends what the Detective found; reference the relevant `det_xx` ID in `based_on` when a finding uses detective context.
51
52	Validation & robustness (REQUIRED whenever a finding is derived, modelled, or predictive — not just descriptive) — don't only state the number; show it can be trusted. Run and record at least one of: a backtest / holdout, a comparison against a naive baseline, a sanity check, or a sensitivity/robustness check — plus what would falsify the claim. Make this its own `ana_xx` finding (a validation finding) so the Editor can give "why believe this" its own beat. A predictive/derived headline with no validation is incomplete.
53
54	- State the level it validates vs the level the headline claims. A validation often confirms skill at a different granularity than the headline sells — e.g. the backtest validates a per-event/per-match outcome, but the headline is an aggregate/tournament-level probability the backtest never directly tests. The validation finding MUST say, in plain words, what level it validates and what level the lead claim is at, and name the gap when they differ. Don't let an Editor read "validated" and assume the headline figure itself was validated.
55	- Record comparability when benchmarking against an external number. When you compare the model to an outside benchmark or forecast (often a `det_xx`), record its `timepoint`, `method`, and `sample_size` alongside the comparison so the Editor doesn't oversell a "disagreement" that is partly a snapshot artefact (a different as-of date, a different method, or a tiny sample) rather than a real divergence.
56
57	Client model (for the interactive centerpiece + any model/derived/predictive finding) — beyond `data_table`, emit a `client_model` so an in-browser explorable can let the reader RE-RUN it: the coefficients/params + a small PURE-JS function (and a compact data slice if needed) that recomputes the result client-side from reader inputs. Keep it cheap enough to run live (closed-form, or a few-thousand-iteration simulation that finishes <1s). Save it to `code/` (e.g. `code/client_model.js`) and reference it from the finding so the Programmer can inline it. See `../../frontend-design-pro/references/interaction_playbook.json` → `recipes.explorable_recompute`.
58
59	### 4. Save all

People who liked this also viewed

6 picks

Type

analyst

146stars19forks

View on GitHub

How to install analyst?

qinghonglin/data2story-skill/analyst

$npx -y skills add QinghongLin/data2story-skill --skill analyst

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Analyst
2
3	Your job is completeness, not curation. List every analysis this dataset can support, grounded in the context the Detective found. You are not deciding what story to tell — that is the Editor's job. You are cataloguing what the data contains.
4
5	## Setup
6
7	- `DATA_DIR` = first argument
8	- `PROJECT_DIR` = second argument
9	- Read `PROJECT_DIR/detective.json` before starting — it tells you what matters in this domain
10	- Also read `PROJECT_DIR/scout.json` if present — surface any `live_status[]` entries as display-only context (cite the dated source; never feed live status to a forecasting/training model)
11	- Outputs: `PROJECT_DIR/code/*.py` (analysis scripts), `PROJECT_DIR/analyst.json`
12
13	## Steps
14
15	### 1. Dataset Profile
16
17	Run code to compute:
18	- File(s), format, row count, column count
19	- What one row represents
20	- Time range, geographic scope
21	- Missing value counts per column
22	- Cardinality of categorical columns
23
24	### 2. Field Inventory
25
26	For every column:
27	- Name, inferred meaning, data type
28	- Sample values
29	- Noteworthy distributions or quirks
30
31	### 3. All Possible Analyses
32
33	Run actual code (Python/Bash) for every applicable category below.
34	Record the actual numbers — not descriptions of what could be computed.
35
36	Distributions — value counts for every categorical field; histogram buckets for every numeric field; null/missing rates.
37
38	Rankings — top and bottom N for every meaningful dimension; concentration (what % of outcomes does the top 10% account for?).
39
40	Group Comparisons — every categorical field as a grouping variable against every numeric/outcome field; note effect size, not just direction.
41
42	Correlations & Relationships — pairwise relationships between numeric fields; categorical interactions (e.g. A × B → outcome).
43
44	Trends & Sequences — time-based patterns if a date/order field exists; first vs. last, early vs. late.
45
46	Anomalies — values more than 2 SD from mean; unexpected zeros, near-perfect concentrations, impossible combinations.
47
48	Experiment-specific — if this is a study/survey: check for order effects, experimenter effects, condition imbalances.
49
50	Context-informed — use `detective.json` items to run any comparisons that have external benchmarks; flag where the data confirms, contradicts, or extends what the Detective found; reference the relevant `det_xx` ID in `based_on` when a finding uses detective context.
51
52	Validation & robustness (REQUIRED whenever a finding is derived, modelled, or predictive — not just descriptive) — don't only state the number; show it can be trusted. Run and record at least one of: a backtest / holdout, a comparison against a naive baseline, a sanity check, or a sensitivity/robustness check — plus what would falsify the claim. Make this its own `ana_xx` finding (a validation finding) so the Editor can give "why believe this" its own beat. A predictive/derived headline with no validation is incomplete.
53
54	- State the level it validates vs the level the headline claims. A validation often confirms skill at a different granularity than the headline sells — e.g. the backtest validates a per-event/per-match outcome, but the headline is an aggregate/tournament-level probability the backtest never directly tests. The validation finding MUST say, in plain words, what level it validates and what level the lead claim is at, and name the gap when they differ. Don't let an Editor read "validated" and assume the headline figure itself was validated.
55	- Record comparability when benchmarking against an external number. When you compare the model to an outside benchmark or forecast (often a `det_xx`), record its `timepoint`, `method`, and `sample_size` alongside the comparison so the Editor doesn't oversell a "disagreement" that is partly a snapshot artefact (a different as-of date, a different method, or a tiny sample) rather than a real divergence.
56
57	Client model (for the interactive centerpiece + any model/derived/predictive finding) — beyond `data_table`, emit a `client_model` so an in-browser explorable can let the reader RE-RUN it: the coefficients/params + a small PURE-JS function (and a compact data slice if needed) that recomputes the result client-side from reader inputs. Keep it cheap enough to run live (closed-form, or a few-thousand-iteration simulation that finishes <1s). Save it to `code/` (e.g. `code/client_model.js`) and reference it from the finding so the Programmer can inline it. See `../../frontend-design-pro/references/interaction_playbook.json` → `recipes.explorable_recompute`.
58
59	### 4. Save all

People who liked this also viewed

6 picks

Type