autoresearch

Name: autoresearch
Author: PaulRBerg

68stars3forks

How to install autoresearch?

paulrberg/agent-skills/autoresearch

$npx -y skills add PaulRBerg/agent-skills --skill autoresearch

Installs into the current project.

Summary

Use for autoresearch or "optimize X overnight/in a loop"; sets up bounded iterative trials for a measurable optimization target.

Files · 1

View on GitHub

SKILL.md

1	# Autoresearch
2
3	Run isolated experiments, measure them consistently, keep verified improvements, and stop on explicit resource or
4	convergence limits.
5
6	## Session Contract
7
8	Resolve the objective, primary metric and direction, benchmark and correctness commands, allowed/off-limits paths,
9	run/runtime/command/cost/regression limits, convergence window, and reporting cadence before the baseline. Infer safe
10	facts from the request and repository; ask only when a missing choice changes the experiment.
11
12	Defaults: 20 runs, two hours wall time, 10 minutes per benchmark, five minutes per correctness check, no new paid API
13	spend, and convergence after five consecutive valid runs without a new retained best. Explicit `--max-runs` and
14	`--max-runtime` values are hard limits.
15
16	## Isolation
17
18	Prefer a dedicated branch in a separate Git worktree. Record its starting commit, path, initial status, allowed paths,
19	and session files in `autoresearch.md`. If isolation is unavailable, require a clean worktree or explicit authorization
20	to share it.
21
22	Never run repository-wide clean, checkout, stash, or reset commands. Revert only paths changed by the current experiment
23	from its recorded pre-run state, and remove only newly created in-scope paths. Preserve unrelated files and all session
24	evidence.
25
26	## Session Module
27
28	Create `autoresearch.md`, deterministic `autoresearch.sh`, optional `autoresearch.checks.sh`, append-only
29	`autoresearch.jsonl`, and optional `autoresearch.ideas.md`. Resolve the module from this `SKILL.md` and initialize the
30	JSONL before the baseline:
31
32	```sh
33	uv run "<skill-dir>/scripts/autoresearch-session.py" init \
34	--file autoresearch.jsonl --metric <name> --direction <higher\|lower> \
35	--max-runs <n> --max-runtime-seconds <seconds> \
36	--max-cost <amount> --convergence-runs <n>
37	```
38
39	The first config record declares `direction`. Record each completed attempt only after the agent assigns its status:
40
41	```sh
42	uv run "<skill-dir>/scripts/autoresearch-session.py" record \
43	--file autoresearch.jsonl --metric <number> \
44	--status <keep\|discard\|crash\|checks_failed> \
45	[--commit <id>] [--description <text>] \
46	[--elapsed-seconds <n>] [--estimated-cost <amount>]
47	```
48
49	Zero and negative metrics are valid values. The agent owns `keep` versus `discard`; the module validates records and
50	uses the declared direction. When the primary metric changes, pass `--metric-name <new>` and
51	`--direction <higher\|lower>` on the first new record. The module appends a new segment config.
52
53	Use `status --format json` for best/delta/MAD/confidence, counts, convergence, budgets, and exact progress rendering:
54
55	```sh
56	uv run "<skill-dir>/scripts/autoresearch-session.py" status --file autoresearch.jsonl
57	```
58
59	`scripts/confidence.sh [jsonl]` and `scripts/summary.sh [jsonl]` remain compatibility adapters. Malformed records or
60	violated invariants fail; noisy, equivalent, or agent-discarded results are reported facts, not helper failures.
61
62	## Experiment Loop
63
64	1. Inspect all in-scope source plus relevant tests or profiles. Create isolation and session files, then record an
65	unchanged baseline.
66	2. Before each run, snapshot allowed paths. Choose one focused hypothesis, implement it, and run the benchmark within
67	its timeout.
68	3. Parse the declared metric. Missing metrics, crashes, timeouts, and failed correctness checks cannot be improvements.
69	4. Run correctness checks for every candidate the agent might retain.
70	5. Use the session status plus repeated measurements to judge noise or equivalence. Keep only a verified improvement
71	within all hard constraints; prefer simpler code when results are equivalent. Otherwise perform the scoped revert.
72	6. Append the agent-assigned record and update `autoresearch.md` when evidence changes the retained best or rules out an
73	approach.
74	7. Incorporate user steering between completed runs. Stop at the first hard limit, user interruption, satisfied target,
75	or helper-reported convergence.
76	8. Read `references/loop-rules.md` only for ambiguous keep/discard judgment, noise handling, backlog maintenance, or
77	thrash recovery.
78
79	## Progress and Completion
80
81	Send sparse updates at the baseline, every five settled runs or material best change, and the final stop. Render the
82	module's exact bar, counts, metrics, budgets, and convergence facts; never infer progress from time or activity. Include
83	the next agent-chosen hypothesis without recording it as settled work.
84
85	Finish with `### 🏁 Autoresearch complete — <stop reason>`, baseline/best/delta/confidence, status counts, kept-file
86	tree, exact checks, worktree/branch, and remaining cleanup or integration. Keep `METRIC` lines, JSONL, commands, and
87	diagnostics undecorated. A resource limit is not converg

People who liked this also viewed

6 picks

Type

find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 20262.6M27k
grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026633k182k
vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026573k29k
improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026522k182k
setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026454k182k
triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026424k182k

autoresearch

68stars3forks

View on GitHub

How to install autoresearch?

paulrberg/agent-skills/autoresearch

$npx -y skills add PaulRBerg/agent-skills --skill autoresearch

Installs into the current project.

Summary

Use for autoresearch or "optimize X overnight/in a loop"; sets up bounded iterative trials for a measurable optimization target.

Files · 1

View on GitHub

SKILL.md

1	# Autoresearch
2
3	Run isolated experiments, measure them consistently, keep verified improvements, and stop on explicit resource or
4	convergence limits.
5
6	## Session Contract
7
8	Resolve the objective, primary metric and direction, benchmark and correctness commands, allowed/off-limits paths,
9	run/runtime/command/cost/regression limits, convergence window, and reporting cadence before the baseline. Infer safe
10	facts from the request and repository; ask only when a missing choice changes the experiment.
11
12	Defaults: 20 runs, two hours wall time, 10 minutes per benchmark, five minutes per correctness check, no new paid API
13	spend, and convergence after five consecutive valid runs without a new retained best. Explicit `--max-runs` and
14	`--max-runtime` values are hard limits.
15
16	## Isolation
17
18	Prefer a dedicated branch in a separate Git worktree. Record its starting commit, path, initial status, allowed paths,
19	and session files in `autoresearch.md`. If isolation is unavailable, require a clean worktree or explicit authorization
20	to share it.
21
22	Never run repository-wide clean, checkout, stash, or reset commands. Revert only paths changed by the current experiment
23	from its recorded pre-run state, and remove only newly created in-scope paths. Preserve unrelated files and all session
24	evidence.
25
26	## Session Module
27
28	Create `autoresearch.md`, deterministic `autoresearch.sh`, optional `autoresearch.checks.sh`, append-only
29	`autoresearch.jsonl`, and optional `autoresearch.ideas.md`. Resolve the module from this `SKILL.md` and initialize the
30	JSONL before the baseline:
31
32	```sh
33	uv run "<skill-dir>/scripts/autoresearch-session.py" init \
34	--file autoresearch.jsonl --metric <name> --direction <higher\|lower> \
35	--max-runs <n> --max-runtime-seconds <seconds> \
36	--max-cost <amount> --convergence-runs <n>
37	```
38
39	The first config record declares `direction`. Record each completed attempt only after the agent assigns its status:
40
41	```sh
42	uv run "<skill-dir>/scripts/autoresearch-session.py" record \
43	--file autoresearch.jsonl --metric <number> \
44	--status <keep\|discard\|crash\|checks_failed> \
45	[--commit <id>] [--description <text>] \
46	[--elapsed-seconds <n>] [--estimated-cost <amount>]
47	```
48
49	Zero and negative metrics are valid values. The agent owns `keep` versus `discard`; the module validates records and
50	uses the declared direction. When the primary metric changes, pass `--metric-name <new>` and
51	`--direction <higher\|lower>` on the first new record. The module appends a new segment config.
52
53	Use `status --format json` for best/delta/MAD/confidence, counts, convergence, budgets, and exact progress rendering:
54
55	```sh
56	uv run "<skill-dir>/scripts/autoresearch-session.py" status --file autoresearch.jsonl
57	```
58
59	`scripts/confidence.sh [jsonl]` and `scripts/summary.sh [jsonl]` remain compatibility adapters. Malformed records or
60	violated invariants fail; noisy, equivalent, or agent-discarded results are reported facts, not helper failures.
61
62	## Experiment Loop
63
64	1. Inspect all in-scope source plus relevant tests or profiles. Create isolation and session files, then record an
65	unchanged baseline.
66	2. Before each run, snapshot allowed paths. Choose one focused hypothesis, implement it, and run the benchmark within
67	its timeout.
68	3. Parse the declared metric. Missing metrics, crashes, timeouts, and failed correctness checks cannot be improvements.
69	4. Run correctness checks for every candidate the agent might retain.
70	5. Use the session status plus repeated measurements to judge noise or equivalence. Keep only a verified improvement
71	within all hard constraints; prefer simpler code when results are equivalent. Otherwise perform the scoped revert.
72	6. Append the agent-assigned record and update `autoresearch.md` when evidence changes the retained best or rules out an
73	approach.
74	7. Incorporate user steering between completed runs. Stop at the first hard limit, user interruption, satisfied target,
75	or helper-reported convergence.
76	8. Read `references/loop-rules.md` only for ambiguous keep/discard judgment, noise handling, backlog maintenance, or
77	thrash recovery.
78
79	## Progress and Completion
80
81	Send sparse updates at the baseline, every five settled runs or material best change, and the final stop. Render the
82	module's exact bar, counts, metrics, budgets, and convergence facts; never infer progress from time or activity. Include
83	the next agent-chosen hypothesis without recording it as settled work.
84
85	Finish with `### 🏁 Autoresearch complete — <stop reason>`, baseline/best/delta/confidence, status counts, kept-file
86	tree, exact checks, worktree/branch, and remaining cleanup or integration. Keep `METRIC` lines, JSONL, commands, and
87	diagnostics undecorated. A resource limit is not converg

People who liked this also viewed

6 picks

Type