mathodology-award-gates

Name: mathodology-award-gates
Author: sweetcornna

36stars5forks

How to install mathodology-award-gates?

sweetcornna/mathodology/mathodology-award-gates

$npx -y skills add sweetcornna/mathodology --skill mathodology-award-gates

Installs into the current project.

Summary

Use when running Mathodology award-workflow phase gates, judge panels, structured handoffs, figure QA, or rendered-PDF QA in a contest run.

Files · 1

View on GitHub

SKILL.md

1	# Mathodology Award Gates
2
3	Canonical home for the runtime contracts of the Mathodology award workflow: the
4	structured run blocks, the severity ladder, the judge-panel aggregation rule,
5	the iteration budgets, the run layout, the blind seat protocol, and the QA
6	scripts. Agents and workflows point here instead of redefining these formats.
7
8	Role-specific extra keys (e.g. a coder's deviations note) live in the agent
9	definitions; this skill owns the shared schema every role must satisfy.
10
11	## 1. Structured Run Blocks
12
13	Every specialist ends with a `handoff:` block. Free-text handoffs are rejected.
14	Lint each block with `lint_run.py handoff` (see Scripts); every `artifacts[].path`
15	must resolve under `work/<run-id>/`.
16
17	```yaml
18	handoff:
19	phase: 4
20	agent: mathodology-coder
21	loop: 0 # 0 = first attempt; increments per gate retry
22	status: complete # complete \| partial \| blocked
23	artifacts:
24	- {path: work/<run-id>/outputs/figures/sens.pdf, role: sensitivity}
25	decisions: []
26	assumptions: [] # each: {id: A7, text: ..., evidence: ...\|assumed, sensitivity_plan: ...}
27	evidence: []
28	commands: [] # exact rerun commands
29	weaknesses: []
30	questions: [] # empty unless contest-critical
31	critic_focus: []
32	```
33
34	The critic writes a `gate:` block per phase. `verdict: fail` on any unresolved
35	`blocker`/`high`. Lint with `lint_run.py gate`.
36
37	```yaml
38	gate:
39	phase: 4
40	loop: 0
41	verdict: pass # pass \| fail
42	issues:
43	- {severity: high, summary: ..., artifact: ..., required_fix: ..., owner: mathodology-coder}
44	evidence_checked: []
45	missing_evidence: []
46	```
47
48	Each Phase-7 judge seat returns one `scorecard:` block. Weights sum to 1.0;
49	scores are 0-100. Lint with `lint_run.py scorecard`.
50
51	```yaml
52	scorecard:
53	contest: MCM
54	target_tier: outstanding
55	seat: A # A \| B \| C
56	round: 1
57	criteria: # one row per criterion; weights sum to 1.0, scores 0-100
58	- {name: modeling, weight: 0.4, score: 82}
59	- {name: results, weight: 0.35, score: 80}
60	- {name: writing, weight: 0.25, score: 88}
61	weighted_total: 82.8
62	implied_tier: meritorious
63	fix_one_thing: "..."
64	ranked_gaps: []
65	do_not_regress: []
66	```
67
68	When a budget is exhausted the lead emits a `decision_memo:` and stops for a
69	human decision (never silently continues). Lint with `lint_run.py memo`.
70
71	```yaml
72	decision_memo:
73	phase: 7
74	budget_spent: {loops: 2, cap: 2}
75	unresolved: [] # remaining issues with severity
76	options: # 2-3 options, each {option, consequence, recommended: bool}
77	- {option: ..., consequence: ..., recommended: true}
78	```
79
80	## 2. Severity Ladder
81
82	- `blocker`: violates contest rules, breaks prompt coverage, invalidates the model, prevents reproduction, or makes submission unsafe.
83	- `high`: likely to lower award level unless fixed, including sparse result presentation or visible figure/table rendering defects in a paper-first contest.
84	- `medium`: should be fixed or explicitly accepted with rationale.
85	- `low`: polish or minor clarity issue that does not affect correctness, scoring, reproducibility, or submission validity.
86
87	No `blocker` or `high` may remain before advancing a phase. `medium` needs an
88	owner, a fix plan, or an explicit, rationale-backed risk acceptance.
89
90	## 3. Judge Aggregation And Thresholds
91
92	Run the panel with `lint_run.py aggregate <scorecard files> --target <tier>`.
93	The panel PASSES only when all of:
94
95	- (a) every seat's `implied_tier` is at or above the target tier;
96	- (b) the minimum seat `weighted_total` clears the target's total threshold;
97	- (c) no single criterion, in any seat, falls below the target's floor.
98
99	Two seats differing by more than 20 on one criterion is an evidence conflict:
100	it is surfaced and adjudicated by the lead, never averaged away, and blocks a
101	clean pass until resolved.
102
103	\| Target tier \| Total >= \| Criterion floor >= \|
104	\|---\|---\|---\|
105	\| Outstanding / 国一 \| 85 \| 70 \|
106	\| Finalist / 国一边缘 \| 80 \| 65 \|
107	\| Meritorious / 国二 \| 75 \| 60 \|
108
109	Calibrate against real rarity: Outstanding is roughly the top 1-2%, 国一 roughly
110	the top 5-8%. Do not inflate scores to force a pass.
111
112	## 4. Iteration Budgets
113
114	- Each per-phase critic gate: at most 2 fix loops (3 evaluations total).
115	- Phase 7: at most 2 re-score rounds.
116	- Whole run: capped at 8 fix loops across all phases.
117	- Stop early whenever a loop shows no improvement over the previous one.
118	- On exhaustion of any budget: emit a `decision_memo:` and stop for a human.
119
120	## 5. Run Layout
121
122	Every run writes under a single gitignored `work/<run-id>/` tree; every handoff
123	artifact path resolves inside it:
124
125	```text
126	work/<run-id>/
127	phase-logs/
128	gates/ # gates/phase-<n>-loop-<k>.yaml
129	scorecards/ # scorecards/p

People who liked this also viewed

6 picks

Type

hyperframes-cliUse the HyperFrames CLI development loop: init, add, catalog, capture, lint, check, snapshot, compare, grade-compare, preview, play, present, beats, keyframes, single or batch render, publish, cloud, SkillsJul 2026262k37k
request-refactor-planSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026165k181k
analyze-projectRigor Analyze / Rigor Audit read-only skill for deep learning research repositories. Use when the user wants to read and understand a repository, inspect model structure and training or inference entrSkillsMay 2026161k507
explore-codeRigor Improve implementation leaf skill for auditable candidate implementation in deep learning research repositories. Use when the researcher explicitly authorizes exploratory work on an isolated braSkillsMay 2026161k507
ubiquitous-languageSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026160k181k
scaffold-exercisesSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026149k181k

mathodology-award-gates

36stars5forks

View on GitHub

How to install mathodology-award-gates?

sweetcornna/mathodology/mathodology-award-gates

$npx -y skills add sweetcornna/mathodology --skill mathodology-award-gates

Installs into the current project.

Summary

Use when running Mathodology award-workflow phase gates, judge panels, structured handoffs, figure QA, or rendered-PDF QA in a contest run.

Files · 1

View on GitHub

SKILL.md

1	# Mathodology Award Gates
2
3	Canonical home for the runtime contracts of the Mathodology award workflow: the
4	structured run blocks, the severity ladder, the judge-panel aggregation rule,
5	the iteration budgets, the run layout, the blind seat protocol, and the QA
6	scripts. Agents and workflows point here instead of redefining these formats.
7
8	Role-specific extra keys (e.g. a coder's deviations note) live in the agent
9	definitions; this skill owns the shared schema every role must satisfy.
10
11	## 1. Structured Run Blocks
12
13	Every specialist ends with a `handoff:` block. Free-text handoffs are rejected.
14	Lint each block with `lint_run.py handoff` (see Scripts); every `artifacts[].path`
15	must resolve under `work/<run-id>/`.
16
17	```yaml
18	handoff:
19	phase: 4
20	agent: mathodology-coder
21	loop: 0 # 0 = first attempt; increments per gate retry
22	status: complete # complete \| partial \| blocked
23	artifacts:
24	- {path: work/<run-id>/outputs/figures/sens.pdf, role: sensitivity}
25	decisions: []
26	assumptions: [] # each: {id: A7, text: ..., evidence: ...\|assumed, sensitivity_plan: ...}
27	evidence: []
28	commands: [] # exact rerun commands
29	weaknesses: []
30	questions: [] # empty unless contest-critical
31	critic_focus: []
32	```
33
34	The critic writes a `gate:` block per phase. `verdict: fail` on any unresolved
35	`blocker`/`high`. Lint with `lint_run.py gate`.
36
37	```yaml
38	gate:
39	phase: 4
40	loop: 0
41	verdict: pass # pass \| fail
42	issues:
43	- {severity: high, summary: ..., artifact: ..., required_fix: ..., owner: mathodology-coder}
44	evidence_checked: []
45	missing_evidence: []
46	```
47
48	Each Phase-7 judge seat returns one `scorecard:` block. Weights sum to 1.0;
49	scores are 0-100. Lint with `lint_run.py scorecard`.
50
51	```yaml
52	scorecard:
53	contest: MCM
54	target_tier: outstanding
55	seat: A # A \| B \| C
56	round: 1
57	criteria: # one row per criterion; weights sum to 1.0, scores 0-100
58	- {name: modeling, weight: 0.4, score: 82}
59	- {name: results, weight: 0.35, score: 80}
60	- {name: writing, weight: 0.25, score: 88}
61	weighted_total: 82.8
62	implied_tier: meritorious
63	fix_one_thing: "..."
64	ranked_gaps: []
65	do_not_regress: []
66	```
67
68	When a budget is exhausted the lead emits a `decision_memo:` and stops for a
69	human decision (never silently continues). Lint with `lint_run.py memo`.
70
71	```yaml
72	decision_memo:
73	phase: 7
74	budget_spent: {loops: 2, cap: 2}
75	unresolved: [] # remaining issues with severity
76	options: # 2-3 options, each {option, consequence, recommended: bool}
77	- {option: ..., consequence: ..., recommended: true}
78	```
79
80	## 2. Severity Ladder
81
82	- `blocker`: violates contest rules, breaks prompt coverage, invalidates the model, prevents reproduction, or makes submission unsafe.
83	- `high`: likely to lower award level unless fixed, including sparse result presentation or visible figure/table rendering defects in a paper-first contest.
84	- `medium`: should be fixed or explicitly accepted with rationale.
85	- `low`: polish or minor clarity issue that does not affect correctness, scoring, reproducibility, or submission validity.
86
87	No `blocker` or `high` may remain before advancing a phase. `medium` needs an
88	owner, a fix plan, or an explicit, rationale-backed risk acceptance.
89
90	## 3. Judge Aggregation And Thresholds
91
92	Run the panel with `lint_run.py aggregate <scorecard files> --target <tier>`.
93	The panel PASSES only when all of:
94
95	- (a) every seat's `implied_tier` is at or above the target tier;
96	- (b) the minimum seat `weighted_total` clears the target's total threshold;
97	- (c) no single criterion, in any seat, falls below the target's floor.
98
99	Two seats differing by more than 20 on one criterion is an evidence conflict:
100	it is surfaced and adjudicated by the lead, never averaged away, and blocks a
101	clean pass until resolved.
102
103	\| Target tier \| Total >= \| Criterion floor >= \|
104	\|---\|---\|---\|
105	\| Outstanding / 国一 \| 85 \| 70 \|
106	\| Finalist / 国一边缘 \| 80 \| 65 \|
107	\| Meritorious / 国二 \| 75 \| 60 \|
108
109	Calibrate against real rarity: Outstanding is roughly the top 1-2%, 国一 roughly
110	the top 5-8%. Do not inflate scores to force a pass.
111
112	## 4. Iteration Budgets
113
114	- Each per-phase critic gate: at most 2 fix loops (3 evaluations total).
115	- Phase 7: at most 2 re-score rounds.
116	- Whole run: capped at 8 fix loops across all phases.
117	- Stop early whenever a loop shows no improvement over the previous one.
118	- On exhaustion of any budget: emit a `decision_memo:` and stop for a human.
119
120	## 5. Run Layout
121
122	Every run writes under a single gitignored `work/<run-id>/` tree; every handoff
123	artifact path resolves inside it:
124
125	```text
126	work/<run-id>/
127	phase-logs/
128	gates/ # gates/phase-<n>-loop-<k>.yaml
129	scorecards/ # scorecards/p

People who liked this also viewed

6 picks

Type