preprocess-imaging

Name: preprocess-imaging
Author: Aperivue

208stars53forks

How to install preprocess-imaging?

aperivue/medsci-skills/preprocess-imaging

$npx -y skills add Aperivue/medsci-skills --skill preprocess-imaging

Installs into the current project.

Summary

Design or audit the data-preparation stage of a medical-imaging model — DICOM/NIfTI intake, resampling and intensity normalisation, and the augmentation plan — so the pipeline is leakage-safe before model-scaffold builds the training repo. Emits a declarative preprocessing manife

Files · 1

View on GitHub

SKILL.md

1	# Preprocess-Imaging Skill
2
3	## Purpose
4
5	This skill designs and audits the data-preparation stage of a medical-imaging model — the stage
6	before a training repo is built — and proves it is leakage-safe by construction. Data leakage
7	enters one step earlier than the split table can see: a normaliser fit on the whole dataset, a
8	data-fitted transform run before the split exists, or a patient whose slices land in more than one
9	partition. Each silently inflates every downstream metric (Kapoor & Narayanan, Patterns 2023;
10	Varoquaux & Cheplygina, npj Digit Med 2022; CLAIM 2024 data items).
11
12	It is the missing first link in the lane: preprocess-imaging (prepare + audit) →
13	`/model-scaffold` (build) → `/model-validation` (validate the split) → `/model-evaluation` +
14	`/analyze-stats` (metrics) → `/write-paper` + `/check-reporting` (publish). It integrates
15	MONAI / TorchIO transforms (referenced in the emitted plan); it does not reimplement them, and it
16	never executes preprocessing on real patient data.
17
18	## When to use
19	- You have a data manifest (one row per image/slice with a patient/subject ID) and want a
20	leakage-safe preprocessing plan + a machine-checkable manifest before scaffolding a model.
21	- You want to audit an existing preprocessing pipeline for data-stage leakage.
22
23	## When NOT to use
24	- Auditing the train/val/test split table itself → `/model-validation` (split-leakage gate).
25	- Building the training repo / model code → `/model-scaffold` (it consumes this manifest).
26	- Choosing the architecture → `/architecture-zoo`.
27	- Held-out metrics / calibration → `/model-evaluation` then `/analyze-stats`.
28	- Reimplementing MONAI / TorchIO transforms → out of scope (this skill wires and audits them).
29
30	## Workflow
31
32	### Phase 1 — Inventory the data and the intended steps
33	Collect: modality (CT / MR / X-ray / US / path), the data manifest (one row per image/slice with a
34	`patient_id`), the intended resample spacing, the intensity transform (fixed HU window vs a fitted
35	z-score / min-max / histogram match), and the augmentation plan. See
36	[`references/preprocessing_guide.md`](references/preprocessing_guide.md) for modality-aware guidance
37	(what normalisation is standard per modality, which augmentations preserve vs break physiology).
38
39	### Phase 2 — Decide fit scope and order (the leakage-safe rules)
40	- Fit dataset-level normalisation on the training split only — never on all/full/test.
41	- Run any data-fitted transform AFTER the split — before the split there is no train/test
42	distinction, so the fit spans partitions.
43	- Prefer per-image (per-sample) normalisation where clinically appropriate: it uses only that
44	image's own statistics and is leakage-free even before the split.
45	- Keep augmentation train-only — augmenting val/test folds undisclosed test-time augmentation
46	into the reported metric.
47	- Split at the patient level, then map slices to their patient's split (never split slices).
48
49	### Phase 3 — Emit the preprocessing manifest
50	Write a declarative JSON manifest that `model-scaffold` consumes and the gate checks:
51
52	```json
53	{
54	"split_seed": 42,
55	"transforms": [
56	{"name": "hu_window", "type": "clip", "fit_scope": "none", "stage": "before_split"},
57	{"name": "train_zscore", "type": "standardize", "fit_scope": "train", "stage": "after_split"},
58	{"name": "flip_rotate", "type": "augmentation", "stage": "after_split", "applies_to": ["train"]}
59	],
60	"split_assignment": [
61	{"patient_id": "P001", "unit_id": "P001_s1", "split": "train"}
62	]
63	}
64	```
65
66	`fit_scope`: `train` (OK) · `all`/`full`/`dataset`/`test` (leak) · `sample`/`per_image`/`none`
67	(not data-fitted, leakage-free). `stage`: `before_split` / `after_split`.
68
69	### Phase 4 — Gate the manifest (deterministic)
70	```bash
71	python3 scripts/check_preprocessing_leakage.py --manifest preproc

People who liked this also viewed

6 picks

Type

preprocess-imaging

208stars53forks

View on GitHub

How to install preprocess-imaging?

aperivue/medsci-skills/preprocess-imaging

$npx -y skills add Aperivue/medsci-skills --skill preprocess-imaging

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Preprocess-Imaging Skill
2
3	## Purpose
4
5	This skill designs and audits the data-preparation stage of a medical-imaging model — the stage
6	before a training repo is built — and proves it is leakage-safe by construction. Data leakage
7	enters one step earlier than the split table can see: a normaliser fit on the whole dataset, a
8	data-fitted transform run before the split exists, or a patient whose slices land in more than one
9	partition. Each silently inflates every downstream metric (Kapoor & Narayanan, Patterns 2023;
10	Varoquaux & Cheplygina, npj Digit Med 2022; CLAIM 2024 data items).
11
12	It is the missing first link in the lane: preprocess-imaging (prepare + audit) →
13	`/model-scaffold` (build) → `/model-validation` (validate the split) → `/model-evaluation` +
14	`/analyze-stats` (metrics) → `/write-paper` + `/check-reporting` (publish). It integrates
15	MONAI / TorchIO transforms (referenced in the emitted plan); it does not reimplement them, and it
16	never executes preprocessing on real patient data.
17
18	## When to use
19	- You have a data manifest (one row per image/slice with a patient/subject ID) and want a
20	leakage-safe preprocessing plan + a machine-checkable manifest before scaffolding a model.
21	- You want to audit an existing preprocessing pipeline for data-stage leakage.
22
23	## When NOT to use
24	- Auditing the train/val/test split table itself → `/model-validation` (split-leakage gate).
25	- Building the training repo / model code → `/model-scaffold` (it consumes this manifest).
26	- Choosing the architecture → `/architecture-zoo`.
27	- Held-out metrics / calibration → `/model-evaluation` then `/analyze-stats`.
28	- Reimplementing MONAI / TorchIO transforms → out of scope (this skill wires and audits them).
29
30	## Workflow
31
32	### Phase 1 — Inventory the data and the intended steps
33	Collect: modality (CT / MR / X-ray / US / path), the data manifest (one row per image/slice with a
34	`patient_id`), the intended resample spacing, the intensity transform (fixed HU window vs a fitted
35	z-score / min-max / histogram match), and the augmentation plan. See
36	[`references/preprocessing_guide.md`](references/preprocessing_guide.md) for modality-aware guidance
37	(what normalisation is standard per modality, which augmentations preserve vs break physiology).
38
39	### Phase 2 — Decide fit scope and order (the leakage-safe rules)
40	- Fit dataset-level normalisation on the training split only — never on all/full/test.
41	- Run any data-fitted transform AFTER the split — before the split there is no train/test
42	distinction, so the fit spans partitions.
43	- Prefer per-image (per-sample) normalisation where clinically appropriate: it uses only that
44	image's own statistics and is leakage-free even before the split.
45	- Keep augmentation train-only — augmenting val/test folds undisclosed test-time augmentation
46	into the reported metric.
47	- Split at the patient level, then map slices to their patient's split (never split slices).
48
49	### Phase 3 — Emit the preprocessing manifest
50	Write a declarative JSON manifest that `model-scaffold` consumes and the gate checks:
51
52	```json
53	{
54	"split_seed": 42,
55	"transforms": [
56	{"name": "hu_window", "type": "clip", "fit_scope": "none", "stage": "before_split"},
57	{"name": "train_zscore", "type": "standardize", "fit_scope": "train", "stage": "after_split"},
58	{"name": "flip_rotate", "type": "augmentation", "stage": "after_split", "applies_to": ["train"]}
59	],
60	"split_assignment": [
61	{"patient_id": "P001", "unit_id": "P001_s1", "split": "train"}
62	]
63	}
64	```
65
66	`fit_scope`: `train` (OK) · `all`/`full`/`dataset`/`test` (leak) · `sample`/`per_image`/`none`
67	(not data-fitted, leakage-free). `stage`: `before_split` / `after_split`.
68
69	### Phase 4 — Gate the manifest (deterministic)
70	```bash
71	python3 scripts/check_preprocessing_leakage.py --manifest preproc

People who liked this also viewed

6 picks

Type