terminal-agent-improvement-loop

Name: terminal-agent-improvement-loop
Author: nowledge-co

496stars37forks

How to install terminal-agent-improvement-loop?

nowledge-co/con-terminal/terminal-agent-improvement-loop

$npx -y skills add nowledge-co/con-terminal --skill terminal-agent-improvement-loop

Installs into the current project.

Summary

Run a benchmark-driven improvement loop for Con's terminal agent. Use when iterating on pane awareness, SSH/tmux behavior, coding-cli flows, benchmark scoring, or progress tracking across many runs.

Files · 1

View on GitHub

SKILL.md

1	# Terminal Agent Improvement Loop
2
3	Use this skill when improving Con as a terminal-native agent, not just fixing a one-off bug.
4
5	Primary references:
6
7	- [`benchmarks/terminal-agent/README.md`](../../benchmarks/terminal-agent/README.md)
8	- [`docs/impl/terminal-agent-benchmark.md`](../../docs/impl/terminal-agent-benchmark.md)
9	- [`docs/impl/terminal-agent-improvement-loop.md`](../../docs/impl/terminal-agent-improvement-loop.md)
10
11	## Workflow
12
13	1. Choose the smallest operator profile that matches the problem.
14	2. Run the benchmark on an idle tab:
15	- `python3 benchmarks/terminal-agent/run.py --profile operator-local-codex-devloop --suite operator`
16	- for repeated clean iterations, prefer `python3 benchmarks/terminal-agent/iterate.py ...`
17	3. Score the resulting run with the matching rubric:
18	- `python3 benchmarks/terminal-agent/score.py --profile ... --record ... --score ...`
19	- or ask the built-in agent to judge the raw record and transcript first:
20	`python3 benchmarks/terminal-agent/judge_llm.py --profile ... --record ... --socket /tmp/con.sock`
21	- then turn that judge artifact into a normal scorecard:
22	`python3 benchmarks/terminal-agent/score.py --profile ... --record ... --judge-file ...`
23	4. Record one short summary, a few lessons, and a few next-focus bullets in the score record.
24	5. Append the scorecard to the tracked improvement log:
25	- `python3 benchmarks/terminal-agent/log_iteration.py --scorecard ... --change "..."`
26	6. Make one focused product change.
27	7. Re-run the same operator profile.
28	8. Generate a report when you need to inspect trend:
29	- `python3 benchmarks/terminal-agent/report.py`
30
31	## Rules
32
33	- Use `strict` suites to protect the floor and `operator` suites to judge real workflows.
34	- Prefer operator profiles that start a fresh conversation and have bounded step timeouts.
35	- Prefer typed control-plane improvements over prompt-only fixes.
36	- Do not overfit to a single benchmark phrase or one host layout.
37	- Keep unknowns honest when the backend cannot prove more.
38	- When benchmark infra changes, say whether the product improved or the measurement improved.
39	- Keep iteration notes concise and comparable across runs.
40	- Keep `docs/impl/terminal-agent-improvement-log.md` useful to a human reader; it should explain what changed, not just repeat the numeric score.
41	- If you use the LLM judge, feed it the raw record and transcript, not only the generated report. The report is a summary, not primary evidence.

People who liked this also viewed

6 picks

Type

terminal-agent-improvement-loop

496stars37forks

View on GitHub

How to install terminal-agent-improvement-loop?

nowledge-co/con-terminal/terminal-agent-improvement-loop

$npx -y skills add nowledge-co/con-terminal --skill terminal-agent-improvement-loop

Installs into the current project.

Summary

Run a benchmark-driven improvement loop for Con's terminal agent. Use when iterating on pane awareness, SSH/tmux behavior, coding-cli flows, benchmark scoring, or progress tracking across many runs.

Files · 1

View on GitHub

SKILL.md

1	# Terminal Agent Improvement Loop
2
3	Use this skill when improving Con as a terminal-native agent, not just fixing a one-off bug.
4
5	Primary references:
6
7	- [`benchmarks/terminal-agent/README.md`](../../benchmarks/terminal-agent/README.md)
8	- [`docs/impl/terminal-agent-benchmark.md`](../../docs/impl/terminal-agent-benchmark.md)
9	- [`docs/impl/terminal-agent-improvement-loop.md`](../../docs/impl/terminal-agent-improvement-loop.md)
10
11	## Workflow
12
13	1. Choose the smallest operator profile that matches the problem.
14	2. Run the benchmark on an idle tab:
15	- `python3 benchmarks/terminal-agent/run.py --profile operator-local-codex-devloop --suite operator`
16	- for repeated clean iterations, prefer `python3 benchmarks/terminal-agent/iterate.py ...`
17	3. Score the resulting run with the matching rubric:
18	- `python3 benchmarks/terminal-agent/score.py --profile ... --record ... --score ...`
19	- or ask the built-in agent to judge the raw record and transcript first:
20	`python3 benchmarks/terminal-agent/judge_llm.py --profile ... --record ... --socket /tmp/con.sock`
21	- then turn that judge artifact into a normal scorecard:
22	`python3 benchmarks/terminal-agent/score.py --profile ... --record ... --judge-file ...`
23	4. Record one short summary, a few lessons, and a few next-focus bullets in the score record.
24	5. Append the scorecard to the tracked improvement log:
25	- `python3 benchmarks/terminal-agent/log_iteration.py --scorecard ... --change "..."`
26	6. Make one focused product change.
27	7. Re-run the same operator profile.
28	8. Generate a report when you need to inspect trend:
29	- `python3 benchmarks/terminal-agent/report.py`
30
31	## Rules
32
33	- Use `strict` suites to protect the floor and `operator` suites to judge real workflows.
34	- Prefer operator profiles that start a fresh conversation and have bounded step timeouts.
35	- Prefer typed control-plane improvements over prompt-only fixes.
36	- Do not overfit to a single benchmark phrase or one host layout.
37	- Keep unknowns honest when the backend cannot prove more.
38	- When benchmark infra changes, say whether the product improved or the measurement improved.
39	- Keep iteration notes concise and comparable across runs.
40	- Keep `docs/impl/terminal-agent-improvement-log.md` useful to a human reader; it should explain what changed, not just repeat the numeric score.
41	- If you use the LLM judge, feed it the raw record and transcript, not only the generated report. The report is a summary, not primary evidence.

People who liked this also viewed

6 picks

Type