$npx -y skills add Aperivue/medsci-skills --skill radiomics-mlProduce or audit a radiomics / tabular clinical-ML study — imaging or clinical features → any classical learner (penalised logistic [LASSO / ridge / elastic-net], SVM, k-NN, naive Bayes, LDA/QDA, decision tree, random forest, gradient boosting [XGBoost / LightGBM / CatBoost], sha
| 1 | # Radiomics / Classical-ML Skill |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | Radiomics + tree-ensemble studies (features → random forest / XGBoost → a clinical outcome) are the |
| 6 | **most common solo-doable clinical-ML workflow** — no GPU, no engineer — and the **most commonly |
| 7 | over-optimistic**: hundreds-to-thousands of features on tens of patients, hyperparameters tuned on the |
| 8 | same folds the performance is reported from, features selected on the whole dataset, unstable features |
| 9 | never filtered, and discrimination (AUC) reported without calibration. This skill produces the pipeline |
| 10 | correctly and audits an existing one, so the clinical result survives review (Lambin 2017; CLEAR; |
| 11 | TRIPOD+AI; PROBAST-AI). |
| 12 | |
| 13 | It sits beside the imaging-DL lane: where `/model-scaffold` builds a deep network, **radiomics-ml** |
| 14 | covers the feature-based classical-ML path. It **integrates** scikit-learn / xgboost / pyradiomics |
| 15 | (referenced in the emitted code); it does not reimplement them and never runs a model on real patient |
| 16 | data. |
| 17 | |
| 18 | ## When to use |
| 19 | - You have a radiomics or clinical/tabular feature table and want to build a random-forest / XGBoost |
| 20 | clinical prediction model that will pass statistical review. |
| 21 | - You want to audit an existing radiomics/ML pipeline for the failure modes below. |
| 22 | |
| 23 | ## When NOT to use |
| 24 | - Deep-learning imaging models → `/architecture-zoo` → `/model-scaffold` → `/model-validation`. |
| 25 | - Classical inferential statistics / a regression model as the estimand → `/analyze-stats`. |
| 26 | - Interpretability of a trained network → `/explainability`. |
| 27 | - Reimplementing scikit-learn / xgboost / pyradiomics → out of scope (this skill wires and audits them). |
| 28 | |
| 29 | ## The failure modes (what the gate enforces) |
| 30 | 1. **No nested CV.** Tuning and reporting on the same folds inflates performance. Use nested CV or a |
| 31 | held-out test set. |
| 32 | 2. **High dimensionality, low events.** Features ≥ events with no dimensionality reduction overfits — |
| 33 | the classic radiomics trap. Apply LASSO / PCA / a stability + redundancy filter. |
| 34 | 3. **Selection outside the fold.** Feature selection fit on the whole dataset leaks the held-out folds. |
| 35 | Nest selection inside each training fold. |
| 36 | 4. **No feature stability.** Radiomics features are unstable across acquisition/segmentation — filter |
| 37 | to reproducible features (ICC / test-retest). |
| 38 | 5. **No calibration.** A clinical prediction model needs calibration (slope/intercept + a flexible |
| 39 | curve), not discrimination alone. |
| 40 | 6. **No external validation.** A single-cohort model needs external / temporal validation for a |
| 41 | clinical claim. |
| 42 | |
| 43 | ## Workflow |
| 44 | |
| 45 | ### Phase 1 — Extract features (integrate, don't reimplement) |
| 46 | For radiomics, extract with **pyradiomics** under reproducible, IBSI-aligned settings (fixed bin width, |
| 47 | resampling, normalisation) — record them. For clinical/tabular data, assemble the feature table with a |
| 48 | patient/subject ID and the outcome. See `references/radiomics_ml_guide.md`. |
| 49 | |
| 50 | ### Phase 2 — Build the pipeline correctly |
| 51 | - **Feature stability** — with test-retest / multi-rater data, keep features with ICC ≥ 0.75. |
| 52 | - **Nested cross-validation** — outer folds estimate performance, inner folds tune; do **feature |
| 53 | selection and scaling inside each training fold** (never on the whole dataset). |
| 54 | - **Dimensionality** — with features ≥ events, use LASSO / a stability+redundancy filter / PCA. |
| 55 | - **Model** — pick from the full classical family for the t |