audit-ml-pipeline

Name: audit-ml-pipeline
Author: probabl-ai

70stars4forks

How to install audit-ml-pipeline?

probabl-ai/skills/audit-ml-pipeline

$npx -y skills add probabl-ai/skills --skill audit-ml-pipeline

Installs into the current project.

Summary

Owns the audit/ folder: one # %% (jupytext percent) Python file per experiment, aligned 1:1 with experiments/NN_<short_name>.py and journal/NN_<short_name>.md, that loads the experiment's skore report read-only and uses bare-last-expression cells whose __repr__ carr

Files · 1

View on GitHub

SKILL.md

1	# Audit ML Pipeline
2
3	Per-experiment, human-readable, agent-executable narrative of a skore
4	report — produced by executing a bare-expression `# %%` file and
5	reading the digest. Read-only against the skore Project.
6
7	## Next-step pointers
8
9	\| Came here from… \| After audit, next is… \|
10	\|---\|---\|
11	\| `iterate-ml-experiment` § 4 record-outcome \| → Read audit digest, fill Status block + JOURNAL row \|
12	\| User free-text ("audit 02", "re-audit 04") \| → Surface metrics to the user; no further dispatch \|
13	\| Re-run of an existing experiment \| → Re-execute the existing audit file; surface diff if metrics changed \|
14
15	The audit is dispatched FIRST in § 4, before any scratch probes.
16	The digest carries the checks summary and the metrics summary — it
17	replaces ad-hoc `scratch/<ts>_inspect_*.py` files for the metric
18	extraction step.
19
20	## Where things live — visual map
21
22	\| Path \| Durability \| Who writes it \| What it holds \|
23	\|---\|---\|---\|---\|
24	\| `audit/<NN>_<short_name>.py` \| Durable (in git) \| This skill, once per experiment \| The bare-expression cells. Source of truth. Can be opened as a notebook in JupyterLab / VS Code for the rich HTML view \|
25	\| `scratch/audit/<stem>/audit.md` \| Ephemeral (gitignored), optional \| `run_cells.py` when given a 2nd arg \| Per-cell markdown digest: source + stdout + last-expression `repr`. Same content as stdout \|
26	\| Stdout from `run_cells.py` \| Captured by the bash tool \| `run_cells.py` (always) \| Streamed digest — the agent reads this directly from the tool output \|
27
28	Mnemonic: `audit/` is source (in git); `scratch/audit/` and
29	stdout are output. Never put the source `.py` under
30	`scratch/audit/`. Never commit anything under `scratch/audit/`.
31
32	## Read-only contract
33
34	The central rule. Surfaced as the first Stop condition below.
35
36	Allowed in `audit/<stem>.py`:
37
38	- `skore.Project(...)` — open the project this experiment wrote to.
39	- `project.summarize()` — list `(key, id)` pairs.
40	- `project.get(id)` — load a specific report by id.
41	- Every `report.*` accessor.
42	- Imports from `<pkg>` (read-only inspection).
43
44	Forbidden in `audit/<stem>.py`:
45
46	- `skore.evaluate(...)` — duplicates the report under the same key
47	and pollutes `summarize()`.
48	- `project.put(...)` — same.
49	- Writes outside `scratch/audit/<stem>/` — no `data/` writes, no
50	`reports/` writes, no edits to `src/<pkg>/`. The audit is a viewer.
51	- Mutation of the loaded `report` that survives the cell (e.g.
52	monkey

People who liked this also viewed

6 picks

Type

audit-ml-pipeline

70stars4forks

View on GitHub

How to install audit-ml-pipeline?

probabl-ai/skills/audit-ml-pipeline

$npx -y skills add probabl-ai/skills --skill audit-ml-pipeline

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Audit ML Pipeline
2
3	Per-experiment, human-readable, agent-executable narrative of a skore
4	report — produced by executing a bare-expression `# %%` file and
5	reading the digest. Read-only against the skore Project.
6
7	## Next-step pointers
8
9	\| Came here from… \| After audit, next is… \|
10	\|---\|---\|
11	\| `iterate-ml-experiment` § 4 record-outcome \| → Read audit digest, fill Status block + JOURNAL row \|
12	\| User free-text ("audit 02", "re-audit 04") \| → Surface metrics to the user; no further dispatch \|
13	\| Re-run of an existing experiment \| → Re-execute the existing audit file; surface diff if metrics changed \|
14
15	The audit is dispatched FIRST in § 4, before any scratch probes.
16	The digest carries the checks summary and the metrics summary — it
17	replaces ad-hoc `scratch/<ts>_inspect_*.py` files for the metric
18	extraction step.
19
20	## Where things live — visual map
21
22	\| Path \| Durability \| Who writes it \| What it holds \|
23	\|---\|---\|---\|---\|
24	\| `audit/<NN>_<short_name>.py` \| Durable (in git) \| This skill, once per experiment \| The bare-expression cells. Source of truth. Can be opened as a notebook in JupyterLab / VS Code for the rich HTML view \|
25	\| `scratch/audit/<stem>/audit.md` \| Ephemeral (gitignored), optional \| `run_cells.py` when given a 2nd arg \| Per-cell markdown digest: source + stdout + last-expression `repr`. Same content as stdout \|
26	\| Stdout from `run_cells.py` \| Captured by the bash tool \| `run_cells.py` (always) \| Streamed digest — the agent reads this directly from the tool output \|
27
28	Mnemonic: `audit/` is source (in git); `scratch/audit/` and
29	stdout are output. Never put the source `.py` under
30	`scratch/audit/`. Never commit anything under `scratch/audit/`.
31
32	## Read-only contract
33
34	The central rule. Surfaced as the first Stop condition below.
35
36	Allowed in `audit/<stem>.py`:
37
38	- `skore.Project(...)` — open the project this experiment wrote to.
39	- `project.summarize()` — list `(key, id)` pairs.
40	- `project.get(id)` — load a specific report by id.
41	- Every `report.*` accessor.
42	- Imports from `<pkg>` (read-only inspection).
43
44	Forbidden in `audit/<stem>.py`:
45
46	- `skore.evaluate(...)` — duplicates the report under the same key
47	and pollutes `summarize()`.
48	- `project.put(...)` — same.
49	- Writes outside `scratch/audit/<stem>/` — no `data/` writes, no
50	`reports/` writes, no edits to `src/<pkg>/`. The audit is a viewer.
51	- Mutation of the loaded `report` that survives the cell (e.g.
52	monkey

People who liked this also viewed

6 picks

Type