verify

Name: verify
Author: Borda

23stars3forks

How to install verify?

borda/ai-rig/verify

$npx -y skills add Borda/AI-Rig --skill verify

Installs into the current project.

Summary

Paper-vs-code consistency audit. After research:scientist implements a method from a paper, verify the implementation matches paper claims across five dimensions — formula matching [F], hyperparameter parity [H], eval protocol [E], notation consistency [N], and citation chain [C]

Files · 1

View on GitHub

SKILL.md

1	<constants>
2
3	```yaml
4	HARD_CUTOFF: 900 # seconds — advisory; Agent() calls are synchronous and cannot be interrupted mid-flight
5	```
6
7	</constants>
8
9	<objective>
10
11	Paper-vs-code consistency audit. After `research:scientist` implements method from paper, verify implementation matches paper claims. Audits five dimensions — formula matching, hyperparameter parity, eval protocol, notation consistency, citation chain. Emits verification table with match status and severity.
12
13	NOT for: running experiments (use `/research:run`); judging experimental methodology (use `/research:judge`); literature search (use `/research:topic`); general code review (use `/develop:review` (requires `develop` plugin)). Verify audits implementation-vs-paper fidelity only — does not evaluate whether paper's claims are valid.
14
15	</objective>
16
17	<workflow>
18
19	## Agent Resolution
20
21	`research:scientist` same plugin as this skill — no fallback needed if research plugin installed. Scientist handles all five audit dimensions in single spawn to preserve cross-dimension context (e.g., notation inconsistency explaining formula mismatch needs holistic paper understanding).
22
23	## Verify Mode (Steps V1–V6)
24
25	Triggered by `verify <paper>` where `<paper>` is PDF path, arXiv URL, or multi-line quoted text.
26
27	Task tracking: create tasks for V1, V2, V3, V4, V5, V6 at start — before any tool calls.
28
29	### Step V1: Parse paper input
30
31	Input resolution (priority order):
32
33	1. Path ending `.pdf` — read via Read tool (use `pages: "1-20"` for large PDFs; iterate with subsequent page ranges if needed — max 20 pages per Read call)
34	2. URL matching `arxiv.org` — convert `abs/<id>` to `https://arxiv.org/pdf/<id>` for actual content fetching (e.g., `ARXIV_URL="${ARXIV_URL//arxiv.org\/abs\//arxiv.org\/pdf\/}"`); also fetch abstract page for metadata. Use WebFetch (`timeout: 30000`).
35	3. URL matching `*.pdf` or `doi.org` — WebFetch (`timeout: 30000`)
36	4. Multi-line quoted text block — treat as literal paper content
37	5. No paper argument — stop: `"No paper provided. Usage: /research:verify <paper.pdf\|arxiv-url\|'pasted text'> [--scope <glob>]"`
38
39	From paper content, extract:
40
41	- Header: title, authors, year (for report)
42	- Claims table: each claim = `{id, section, claim_text, type}` where type is one of: `formula`, `hyperparameter`, `eval`, `architecture`, `result`
43	- Focus on: equations with concrete terms, specific hyperparameter values, evaluation protocols (metric names, split names, preprocessing steps), architectural specifics, reported numeric results
44
45	Unsupported flag check: load and follow the protocol below. Supported flags for this skill: `--scope`, `--program`, `--strict`, `--dim`, `--codemap`, `--no-codemap`.
46	```bash
47	# loads: unsupported-flag-protocol.md
48	_RESEARCH_SHARED=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" 2>/dev/null) # timeout: 5000
49	[ -z "$_RESEARCH_SHARED" ] && { echo "! Plugin path resolution failed"; exit 1; }
50	cat "$_RESEARCH_SHARED/unsupported-flag-protocol.md"
51	```
52
53	Codemap auto-detection — structural context (blast-radius, importers, coverage) for the audited codebase; on by default when codemap installed + index found. `--no-codemap` opts out; `--codemap` is strict (fail if unavailable). Note: `--codemap` is independent of `--strict` (audit strictness).
54
55	```bash
56	# timeout: 5000
57	export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
58	CODEMAP_RAW=auto
59	[[ " $ARGUMENTS " == " --no-codemap " ]] && CODEMAP_RAW=off
60	[[ " $ARGUMENTS " == " --codemap " ]] && [[ " $ARGUMENTS " != " --no-codemap " ]] && CODEMAP_RAW=strict
61	CODEMAP_ENABLED=$("${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/codemap-resolve" "$CODEMAP_RAW")
62	if [ $? -ne 0 ]; then
63	[ "$CODEMAP_RAW" = "strict" ] && { echo "! BLOCKED — --codemap (strict) but codemap unavailable; run /codemap:scan-codebase or install codemap plugin"; exit 1; }
64	CODEMAP_ENABLED=false
65	fi
66	echo "$CODEMAP_ENABLED" > "${TMPDIR:-/tmp}/research-verify-codemap-enabled-${CSID}"
67	```
68
69	> loads: codemap-gates.md
70
71	When `CODEMAP_RAW` ≠ `off`:
72	```bash
73	_RESEARCH_SHARED=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" 2>/dev/null) # timeout: 5000
74	cat "$_RESEARCH_SHARED/codemap-gates.md"
75	```
76	Follow Gate A and Gate B.
77
78	Pre-compute run directory — persist `RUN_DIR` and `OUT` to temp fil

People who liked this also viewed

6 picks

Type

verify

23stars3forks

View on GitHub

How to install verify?

borda/ai-rig/verify

$npx -y skills add Borda/AI-Rig --skill verify

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	<constants>
2
3	```yaml
4	HARD_CUTOFF: 900 # seconds — advisory; Agent() calls are synchronous and cannot be interrupted mid-flight
5	```
6
7	</constants>
8
9	<objective>
10
11	Paper-vs-code consistency audit. After `research:scientist` implements method from paper, verify implementation matches paper claims. Audits five dimensions — formula matching, hyperparameter parity, eval protocol, notation consistency, citation chain. Emits verification table with match status and severity.
12
13	NOT for: running experiments (use `/research:run`); judging experimental methodology (use `/research:judge`); literature search (use `/research:topic`); general code review (use `/develop:review` (requires `develop` plugin)). Verify audits implementation-vs-paper fidelity only — does not evaluate whether paper's claims are valid.
14
15	</objective>
16
17	<workflow>
18
19	## Agent Resolution
20
21	`research:scientist` same plugin as this skill — no fallback needed if research plugin installed. Scientist handles all five audit dimensions in single spawn to preserve cross-dimension context (e.g., notation inconsistency explaining formula mismatch needs holistic paper understanding).
22
23	## Verify Mode (Steps V1–V6)
24
25	Triggered by `verify <paper>` where `<paper>` is PDF path, arXiv URL, or multi-line quoted text.
26
27	Task tracking: create tasks for V1, V2, V3, V4, V5, V6 at start — before any tool calls.
28
29	### Step V1: Parse paper input
30
31	Input resolution (priority order):
32
33	1. Path ending `.pdf` — read via Read tool (use `pages: "1-20"` for large PDFs; iterate with subsequent page ranges if needed — max 20 pages per Read call)
34	2. URL matching `arxiv.org` — convert `abs/<id>` to `https://arxiv.org/pdf/<id>` for actual content fetching (e.g., `ARXIV_URL="${ARXIV_URL//arxiv.org\/abs\//arxiv.org\/pdf\/}"`); also fetch abstract page for metadata. Use WebFetch (`timeout: 30000`).
35	3. URL matching `*.pdf` or `doi.org` — WebFetch (`timeout: 30000`)
36	4. Multi-line quoted text block — treat as literal paper content
37	5. No paper argument — stop: `"No paper provided. Usage: /research:verify <paper.pdf\|arxiv-url\|'pasted text'> [--scope <glob>]"`
38
39	From paper content, extract:
40
41	- Header: title, authors, year (for report)
42	- Claims table: each claim = `{id, section, claim_text, type}` where type is one of: `formula`, `hyperparameter`, `eval`, `architecture`, `result`
43	- Focus on: equations with concrete terms, specific hyperparameter values, evaluation protocols (metric names, split names, preprocessing steps), architectural specifics, reported numeric results
44
45	Unsupported flag check: load and follow the protocol below. Supported flags for this skill: `--scope`, `--program`, `--strict`, `--dim`, `--codemap`, `--no-codemap`.
46	```bash
47	# loads: unsupported-flag-protocol.md
48	_RESEARCH_SHARED=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" 2>/dev/null) # timeout: 5000
49	[ -z "$_RESEARCH_SHARED" ] && { echo "! Plugin path resolution failed"; exit 1; }
50	cat "$_RESEARCH_SHARED/unsupported-flag-protocol.md"
51	```
52
53	Codemap auto-detection — structural context (blast-radius, importers, coverage) for the audited codebase; on by default when codemap installed + index found. `--no-codemap` opts out; `--codemap` is strict (fail if unavailable). Note: `--codemap` is independent of `--strict` (audit strictness).
54
55	```bash
56	# timeout: 5000
57	export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
58	CODEMAP_RAW=auto
59	[[ " $ARGUMENTS " == " --no-codemap " ]] && CODEMAP_RAW=off
60	[[ " $ARGUMENTS " == " --codemap " ]] && [[ " $ARGUMENTS " != " --no-codemap " ]] && CODEMAP_RAW=strict
61	CODEMAP_ENABLED=$("${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/codemap-resolve" "$CODEMAP_RAW")
62	if [ $? -ne 0 ]; then
63	[ "$CODEMAP_RAW" = "strict" ] && { echo "! BLOCKED — --codemap (strict) but codemap unavailable; run /codemap:scan-codebase or install codemap plugin"; exit 1; }
64	CODEMAP_ENABLED=false
65	fi
66	echo "$CODEMAP_ENABLED" > "${TMPDIR:-/tmp}/research-verify-codemap-enabled-${CSID}"
67	```
68
69	> loads: codemap-gates.md
70
71	When `CODEMAP_RAW` ≠ `off`:
72	```bash
73	_RESEARCH_SHARED=$(python "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/resolve_shared.py" 2>/dev/null) # timeout: 5000
74	cat "$_RESEARCH_SHARED/codemap-gates.md"
75	```
76	Follow Gate A and Gate B.
77
78	Pre-compute run directory — persist `RUN_DIR` and `OUT` to temp fil

People who liked this also viewed

6 picks

Type