design-study

Name: design-study
Author: Aperivue

208stars53forks

How to install design-study?

aperivue/medsci-skills/design-study

$npx -y skills add Aperivue/medsci-skills --skill design-study

Installs into the current project.

Summary

Study design and validity review for radiology and medical AI research. Identifies analysis unit, cohort logic, leakage risks, comparator design, validation strategy, and reporting guideline fit before drafting or submission.

Files · 1

View on GitHub

SKILL.md

1	# Design-Study Skill
2
3	## Purpose
4
5	This skill pressure-tests whether a study is answerable, interpretable, and defensible before large amounts of drafting or analysis work accumulate.
6
7	Use it when:
8	- a study question is known but the analysis plan is still fluid
9	- the user wants a methods sanity check
10	- a manuscript feels vulnerable to reviewer criticism
11	- a peer review requires explicit methodological diagnosis
12
13	---
14
15	## Communication Rules
16
17	- Communicate with the user in their preferred language.
18	- Use English for statistical, radiologic, and reporting-guideline terminology.
19	- Be direct about validity risks, but always propose the smallest feasible fix first.
20
21	---
22
23	## Core Review Questions
24
25	Always inspect these dimensions:
26
27	1. What is the exact research question?
28	2. What is the analysis unit: patient, lesion, exam, study, phase, report?
29	3. What is the index date or decision point?
30	4. How are inclusion and exclusion criteria applied?
31	5. Is there any information leakage?
32	6. What is the reference standard or endpoint definition?
33	7. What comparator is clinically meaningful?
34	8. What validation strategy is used?
35	9. What uncertainty reporting is required?
36	10. Which reporting guideline best fits?
37	11. Are exposure/outcome/covariate definitions literature-grounded, or invented ad-hoc from the data dictionary? If ad-hoc, defer to `/define-variables` before drafting Methods.
38
39	---
40
41	## Standard Output
42
43	```text
44	## Study Design Review
45	Question: ...
46	Study type: ...
47	Analysis unit: ...
48	Index date / prediction timepoint: ...
49
50	### Strengths
51	- ...
52
53	### Major validity risks
54	1. ...
55	2. ...
56
57	### Minimal fixes
58	- ...
59
60	### Reporting fit
61	- Recommended guideline: ...
62
63	### Decision
64	- Ready for analysis / Needs redesign / Drafting can proceed with limitations
65	```
66
67	---
68
69	## Workflow
70
71	### Phase 1: Reconstruct the study
72
73	Extract from protocol, draft, slides, tables, or notes:
74	- clinical problem
75	- intended use case
76	- population
77	- inputs
78	- outputs
79	- outcome definition
80	- timing of variable availability
81
82	Gate: Present the reconstructed study summary (question, analysis unit, intended use)
83	to the user. Confirm before proceeding — if the reconstruction is wrong, the entire
84	validity review will be misdirected.
85
86	### Phase 2: Check structural validity
87
88	#### A. Analysis unit
89
90	Look for mismatches such as:
91	- patient-level claim from lesion-level analysis
92	- exam-level split with patient overlap
93	- phase-level samples treated as independent
94
95	#### B. Leakage
96
97	Look for:
98	- postoperative features used for preoperative prediction
99	- normalization or thresholding performed before data split
100	- repeated exams across train/test
101	- reader annotations derived from outcome information
102	- input-text contamination for NLP/LLM extraction tasks: if the model input includes report
103	sections such as clinical history, indication, impression, prior diagnosis, or referral text, confirm
104	that those fields do not literally name or strongly imply the target label. If the target is already
105	present in the supplied text, the task is information retrieval under label leakage, not phenotype
106	inference; redesign the input mask, report a sensitivity analysis excluding leaky fields, or reframe the
107	claim.
108	- construct dependence (a predictor that is a definitional component of the outcome). Two cases:
109	(i) mathematical definition — an input that computes the outcome (when the outcome is HOMA-IR =
110	f(fasting insulin, fasting glucose), those two inputs are not independent predictors); (ii)
111	near-tautological composite — a ratio or score built from the outcome's defining components, which
112	shows an inflated, near-circular association. Test: "could this predictor be derived, in whole or
113	part, from the outcome's definition or the same measurement?" If yes, exclude it, or retain it only
114	as a labeled calibration probe rather than a reported discovery.
115
116	#### F. Time origin & survivorship (incident / transition models)
117
118	For any time-to-event or incident/transition design, check before drafting:
119	- Time origin per model. Each incident model starts its at-risk clock at the correct origin. Watch for immortal-time bias (a span in which the event cannot occur, misattributed to one group) and left-truncation / delayed entry (subjects entering the risk set after the origin).
120	- Mediator-ascertainment-window survivorship. A "progressor" / transition label that is conditional on surviving to a later ascertainment (a second scan, a follow-up visit) is survivorship-biased; plan a landmark time or an explicit intermediat

People who liked this also viewed

6 picks

Type

design-study

208stars53forks

View on GitHub

How to install design-study?

aperivue/medsci-skills/design-study

$npx -y skills add Aperivue/medsci-skills --skill design-study

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Design-Study Skill
2
3	## Purpose
4
5	This skill pressure-tests whether a study is answerable, interpretable, and defensible before large amounts of drafting or analysis work accumulate.
6
7	Use it when:
8	- a study question is known but the analysis plan is still fluid
9	- the user wants a methods sanity check
10	- a manuscript feels vulnerable to reviewer criticism
11	- a peer review requires explicit methodological diagnosis
12
13	---
14
15	## Communication Rules
16
17	- Communicate with the user in their preferred language.
18	- Use English for statistical, radiologic, and reporting-guideline terminology.
19	- Be direct about validity risks, but always propose the smallest feasible fix first.
20
21	---
22
23	## Core Review Questions
24
25	Always inspect these dimensions:
26
27	1. What is the exact research question?
28	2. What is the analysis unit: patient, lesion, exam, study, phase, report?
29	3. What is the index date or decision point?
30	4. How are inclusion and exclusion criteria applied?
31	5. Is there any information leakage?
32	6. What is the reference standard or endpoint definition?
33	7. What comparator is clinically meaningful?
34	8. What validation strategy is used?
35	9. What uncertainty reporting is required?
36	10. Which reporting guideline best fits?
37	11. Are exposure/outcome/covariate definitions literature-grounded, or invented ad-hoc from the data dictionary? If ad-hoc, defer to `/define-variables` before drafting Methods.
38
39	---
40
41	## Standard Output
42
43	```text
44	## Study Design Review
45	Question: ...
46	Study type: ...
47	Analysis unit: ...
48	Index date / prediction timepoint: ...
49
50	### Strengths
51	- ...
52
53	### Major validity risks
54	1. ...
55	2. ...
56
57	### Minimal fixes
58	- ...
59
60	### Reporting fit
61	- Recommended guideline: ...
62
63	### Decision
64	- Ready for analysis / Needs redesign / Drafting can proceed with limitations
65	```
66
67	---
68
69	## Workflow
70
71	### Phase 1: Reconstruct the study
72
73	Extract from protocol, draft, slides, tables, or notes:
74	- clinical problem
75	- intended use case
76	- population
77	- inputs
78	- outputs
79	- outcome definition
80	- timing of variable availability
81
82	Gate: Present the reconstructed study summary (question, analysis unit, intended use)
83	to the user. Confirm before proceeding — if the reconstruction is wrong, the entire
84	validity review will be misdirected.
85
86	### Phase 2: Check structural validity
87
88	#### A. Analysis unit
89
90	Look for mismatches such as:
91	- patient-level claim from lesion-level analysis
92	- exam-level split with patient overlap
93	- phase-level samples treated as independent
94
95	#### B. Leakage
96
97	Look for:
98	- postoperative features used for preoperative prediction
99	- normalization or thresholding performed before data split
100	- repeated exams across train/test
101	- reader annotations derived from outcome information
102	- input-text contamination for NLP/LLM extraction tasks: if the model input includes report
103	sections such as clinical history, indication, impression, prior diagnosis, or referral text, confirm
104	that those fields do not literally name or strongly imply the target label. If the target is already
105	present in the supplied text, the task is information retrieval under label leakage, not phenotype
106	inference; redesign the input mask, report a sensitivity analysis excluding leaky fields, or reframe the
107	claim.
108	- construct dependence (a predictor that is a definitional component of the outcome). Two cases:
109	(i) mathematical definition — an input that computes the outcome (when the outcome is HOMA-IR =
110	f(fasting insulin, fasting glucose), those two inputs are not independent predictors); (ii)
111	near-tautological composite — a ratio or score built from the outcome's defining components, which
112	shows an inflated, near-circular association. Test: "could this predictor be derived, in whole or
113	part, from the outcome's definition or the same measurement?" If yes, exclude it, or retain it only
114	as a labeled calibration probe rather than a reported discovery.
115
116	#### F. Time origin & survivorship (incident / transition models)
117
118	For any time-to-event or incident/transition design, check before drafting:
119	- Time origin per model. Each incident model starts its at-risk clock at the correct origin. Watch for immortal-time bias (a span in which the event cannot occur, misattributed to one group) and left-truncation / delayed entry (subjects entering the risk set after the origin).
120	- Mediator-ascertainment-window survivorship. A "progressor" / transition label that is conditional on surviving to a later ascertainment (a second scan, a follow-up visit) is survivorship-biased; plan a landmark time or an explicit intermediat

People who liked this also viewed

6 picks

Type