woz-benchmark

Name: woz-benchmark
Author: WithWoz

196stars23forks

How to install woz-benchmark?

withwoz/wozcode-plugin/woz-benchmark

$npx -y skills add WithWoz/wozcode-plugin --skill woz-benchmark

Installs into the current project.

Summary

Compare WOZCODE vs vanilla Claude Code on the user's codebase — real cost, turn, and time savings. TRIGGER on "compare woz", "how much does woz save", "benchmark woz", "woz vs claude", "show me savings", or /woz-benchmark.

Files · 1

View on GitHub

SKILL.md

1	# WOZCODE Savings Benchmark
2
3	Run a side-by-side comparison of WOZCODE vs vanilla Claude Code on the user's own codebase. Each prompt runs twice against a fresh copy of the repo with `git reset --hard` between runs, so the target MUST be a clean git repo.
4
5	TRIGGER: "compare woz", "how much does woz save", "benchmark woz", "woz vs claude", "show me the savings", "is woz worth it", or `/woz-benchmark`.
6
7	## Prerequisites
8
9	- User logged in to WOZCODE.
10	- Target directory is a git repo with a clean working tree.
11
12	## Steps
13
14	### 1. Gather inputs — BE BRIEF
15
16	Ask for all three in ONE short message (< 10 lines). Do not re-explain what the benchmark does — the user already invoked it.
17
18	1. Target directory — absolute path to a clean git repo to run the test on.
19	2. Prompts — 2–10 real coding tasks. Tell them briefly: "meaty feature/refactor/bugfix work, not one-liners — trivial prompts hide WOZCODE's advantage". If they don't have prompts in mind, offer to suggest some after looking at their repo.
20	3. Environment setup (optional) — one line: "Anything Claude needs already in place (DB seeded, services running, credentials in `.env`)? Skip if the repo is self-contained."
21
22	Do NOT ask about the model. Default to `opus` in the YAML config. Only switch to `sonnet` or `haiku` if the user volunteers a different choice in their answer.
23
24	### 2. Validate the target
25
26	Before writing any config, verify the target is usable:
27
28	```bash
29	test -d <target>
30	git -C <target> rev-parse --git-dir
31	git -C <target> status --porcelain
32	```
33
34	If the directory doesn't exist, isn't a git repo, or has uncommitted changes, STOP and tell the user how to fix it.
35
36	### 3. Write a temporary benchmark config
37
38	Use the Write tool to create a YAML file at `/tmp/woz-benchmark-<timestamp>.yaml` (get the timestamp from `date +%s`). Format:
39
40	```yaml
41	model: opus
42	maxTurns: 15
43	prompts:
44	- "first prompt from the user"
45	- "second prompt from the user"
46	setup:
47	commands:
48	- "curl -L https://example.com/dataset.csv -o data/sample.csv"
49	- "psql $DATABASE_URL -f seed.sql"
50	```
51
52	Omit the entire `setup:` block if the user didn't give any environment setup commands.
53
54	### 4. Run the benchmark
55
56	One-line warning: "This'll take several minutes — each prompt runs twice." Then run:
57
58	```bash
59	node "${CODEX_HOME:-$HOME/.codex}/plugins/wozcode/scripts/benchmark.js" --target <target> --config <yaml-path> --user-env
60	```
61
62	`--user-env` loads the user's project `CLAUDE.md` hierarchy on BOTH sides. Do NOT pass `--screenshots`, `--codex`, `--judge`, or `--trace`.
63
64	### 5. Present the results as a savings report
65
66	The benchmark prints a detailed text report at the end. Relay the full report to the user, then add a clear, sales-oriented savings summary at the top. Compute the deltas from the report's totals.

People who liked this also viewed

6 picks

Type

woz-benchmark

196stars23forks

View on GitHub

How to install woz-benchmark?

withwoz/wozcode-plugin/woz-benchmark

$npx -y skills add WithWoz/wozcode-plugin --skill woz-benchmark

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# WOZCODE Savings Benchmark
2
3	Run a side-by-side comparison of WOZCODE vs vanilla Claude Code on the user's own codebase. Each prompt runs twice against a fresh copy of the repo with `git reset --hard` between runs, so the target MUST be a clean git repo.
4
5	TRIGGER: "compare woz", "how much does woz save", "benchmark woz", "woz vs claude", "show me the savings", "is woz worth it", or `/woz-benchmark`.
6
7	## Prerequisites
8
9	- User logged in to WOZCODE.
10	- Target directory is a git repo with a clean working tree.
11
12	## Steps
13
14	### 1. Gather inputs — BE BRIEF
15
16	Ask for all three in ONE short message (< 10 lines). Do not re-explain what the benchmark does — the user already invoked it.
17
18	1. Target directory — absolute path to a clean git repo to run the test on.
19	2. Prompts — 2–10 real coding tasks. Tell them briefly: "meaty feature/refactor/bugfix work, not one-liners — trivial prompts hide WOZCODE's advantage". If they don't have prompts in mind, offer to suggest some after looking at their repo.
20	3. Environment setup (optional) — one line: "Anything Claude needs already in place (DB seeded, services running, credentials in `.env`)? Skip if the repo is self-contained."
21
22	Do NOT ask about the model. Default to `opus` in the YAML config. Only switch to `sonnet` or `haiku` if the user volunteers a different choice in their answer.
23
24	### 2. Validate the target
25
26	Before writing any config, verify the target is usable:
27
28	```bash
29	test -d <target>
30	git -C <target> rev-parse --git-dir
31	git -C <target> status --porcelain
32	```
33
34	If the directory doesn't exist, isn't a git repo, or has uncommitted changes, STOP and tell the user how to fix it.
35
36	### 3. Write a temporary benchmark config
37
38	Use the Write tool to create a YAML file at `/tmp/woz-benchmark-<timestamp>.yaml` (get the timestamp from `date +%s`). Format:
39
40	```yaml
41	model: opus
42	maxTurns: 15
43	prompts:
44	- "first prompt from the user"
45	- "second prompt from the user"
46	setup:
47	commands:
48	- "curl -L https://example.com/dataset.csv -o data/sample.csv"
49	- "psql $DATABASE_URL -f seed.sql"
50	```
51
52	Omit the entire `setup:` block if the user didn't give any environment setup commands.
53
54	### 4. Run the benchmark
55
56	One-line warning: "This'll take several minutes — each prompt runs twice." Then run:
57
58	```bash
59	node "${CODEX_HOME:-$HOME/.codex}/plugins/wozcode/scripts/benchmark.js" --target <target> --config <yaml-path> --user-env
60	```
61
62	`--user-env` loads the user's project `CLAUDE.md` hierarchy on BOTH sides. Do NOT pass `--screenshots`, `--codex`, `--judge`, or `--trace`.
63
64	### 5. Present the results as a savings report
65
66	The benchmark prints a detailed text report at the end. Relay the full report to the user, then add a clear, sales-oriented savings summary at the top. Compute the deltas from the report's totals.

People who liked this also viewed

6 picks

Type