judge-with-debate

Name: judge-with-debate
Author: NeoLabHQ

893installs132forks

How to install judge-with-debate?

neolabhq/context-engineering-kit/judge-with-debate

$npx -y skills add NeoLabHQ/context-engineering-kit --skill judge-with-debate

Installs into the current project.

Summary

Evaluate solutions through multi-round debate between independent judges until consensus

Files · 1

View on GitHub

SKILL.md

1	# judge-with-debate
2
3	<task>
4	Evaluate solutions through multi-agent debate where independent judges analyze, challenge each other's assessments, and iteratively refine their evaluations until reaching consensus or maximum rounds.
5	</task>
6
7	<context>
8	This command implements the Multi-Agent Debate pattern for high-quality evaluation where multiple perspectives and rigorous argumentation improve assessment accuracy. Unlike single-pass evaluation, debate forces judges to defend their positions with evidence and consider counter-arguments.
9
10	Key benefits:
11
12	- Structured evaluation - Meta-judge produces tailored rubrics and criteria before judging begins
13	- Multiple perspectives - Three independent judges reduce individual bias
14	- Evidence-based debate - Judges defend positions with specific evidence from the solution and evaluation specification
15	- Iterative refinement - Up to 3 debate rounds drive convergence on accurate scores
16	- Shared specification - Meta-judge runs once; all judges across all rounds share the same evaluation specification
17	</context>
18
19	## Pattern: Debate-Based Evaluation
20
21	This command implements iterative multi-judge debate:
22
23	```
24	Phase 0: Setup
25	mkdir -p .specs/reports
26	\|
27	Phase 0.5: Dispatch Meta-Judge
28	Meta-Judge (Opus)
29	\|
30	Evaluation Specification YAML
31	\|
32	Phase 1: Independent Analysis (3 judges in parallel)
33	+- Judge 1 -> {name}.1.md -+
34	Solution +- Judge 2 -> {name}.2.md -+-+
35	+- Judge 3 -> {name}.3.md -+ \|
36	\|
37	Phase 2: Debate Round (iterative) \|
38	Each judge reads others' reports \|
39	\| \|
40	Argue + Defend + Challenge \|
41	(grounded in eval specification) \|
42	\| \|
43	Revise if convinced --------------+
44	\| \|
45	Check consensus \|
46	+- Yes -> Final Report \|
47	+- No -> Next Round ---------+
48	```
49
50	## Process
51
52	### Setup: Create Reports Directory
53
54	Before starting evaluation, ensure the reports directory exists:
55
56	```bash
57	mkdir -p .specs/reports
58	```
59
60	Report naming convention: `.specs/reports/{solution-name}-{YYYY-MM-DD}.[1\|2\|3].md`
61
62	Where:
63	- `{solution-name}` - Derived from solution filename (e.g., `users-api` from `src/api/users.ts`)
64	- `{YYYY-MM-DD}` - Current date
65	- `[1\|2\|3]` - Judge number
66
67	### Phase 0.5: Dispatch Meta-Judge
68
69	Before independent analysis, dispatch a meta-judge agent to generate a tailored evaluation specification. The meta-judge runs ONCE and produces rubrics, checklists, and scoring criteria that ALL judges will use across ALL rounds.
70
71	Meta-judge prompt template:
72
73	```markdown
74	## Task
75
76	Generate an evaluation specification yaml for the following evaluation task. You will produce rubrics, checklists, and scoring criteria that multiple judge agents will use to evaluate the solution through independent analysis and multi-round debate.
77
78	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
79
80	## User Prompt
81	{task description - what the solution was supposed to accomplish}
82
83	## Context
84	{Any relevant context about the solution being evaluated}
85
86	## Artifact Type
87	{code \| documentation \| configuration \| etc.}
88
89	## Evaluation Mode
90	Multi-judge debate with consensus-seeking across rounds
91
92	## Instructions
93	Return only the final evaluation specification YAML in your response.
94	The specification should support both independent analysis and debate-based refinement.
95	```
96
97	Dispatch:
98
99	```
100	Use Task tool:
101	- description: "Meta-judge: generate evaluation specification for {solution-name}"
102	- prompt: {meta-judge prompt}
103	- model: opus
104	- subagent_type: "sadd:meta-judge"
105	```
106
107	Wait for the meta-judge to complete and extract the evaluation specification YAML from its output before proceeding to Phase 1.
108
109	### Phase 1: Independent Analysis
110
111	Launch 3 independent judge agents in parallel (Opus for rigor):
112
113	1. Each judge receives:
114	- Path to solution(s) being evaluated
115	- The meta-judge's evaluation specification YAML
116	- Task description
117	2. Each produces independent assessment saved to `.specs/reports/{solution-name}-{date}.[1\|2\|3].md`
118	3. Reports must include:
119	- Per-criterion scores with evidence
120	- Specific quotes/examples supporting ratings
121	- Overall weighted score
122	- Key strengths and weaknesses
123
124	Key principle: Independence in initial analysis prevents groupthink.
125
126	Prompt template for initial judges:
127
128	```markdown
129	You are Judge {N} evaluating a solution independently against an evaluation specification produced by the meta judge.
130
131	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
132
133	## Solution
134	{path to solution file(s)}
135
136	## Task Description
137	{what the solution was supposed to accomplish}
138
139	## Evaluation Specification
140
141	`

People who liked this also viewed

6 picks

Type

tddSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026501k181k
test-driven-developmentUse when implementing any feature or bugfix, before writing implementation codeSkillsJul 2026175k259k
qaSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026160k181k
verification-before-completionUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidenceSkillsJul 2026153k259k
webapp-testingToolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser lSkillsJul 2026120k163k
playwright-best-practicesUse when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, teSkillsJul 202664k338

judge-with-debate

893installs132forks

View on GitHub

How to install judge-with-debate?

neolabhq/context-engineering-kit/judge-with-debate

$npx -y skills add NeoLabHQ/context-engineering-kit --skill judge-with-debate

Installs into the current project.

Summary

Evaluate solutions through multi-round debate between independent judges until consensus

Files · 1

View on GitHub

SKILL.md

1	# judge-with-debate
2
3	<task>
4	Evaluate solutions through multi-agent debate where independent judges analyze, challenge each other's assessments, and iteratively refine their evaluations until reaching consensus or maximum rounds.
5	</task>
6
7	<context>
8	This command implements the Multi-Agent Debate pattern for high-quality evaluation where multiple perspectives and rigorous argumentation improve assessment accuracy. Unlike single-pass evaluation, debate forces judges to defend their positions with evidence and consider counter-arguments.
9
10	Key benefits:
11
12	- Structured evaluation - Meta-judge produces tailored rubrics and criteria before judging begins
13	- Multiple perspectives - Three independent judges reduce individual bias
14	- Evidence-based debate - Judges defend positions with specific evidence from the solution and evaluation specification
15	- Iterative refinement - Up to 3 debate rounds drive convergence on accurate scores
16	- Shared specification - Meta-judge runs once; all judges across all rounds share the same evaluation specification
17	</context>
18
19	## Pattern: Debate-Based Evaluation
20
21	This command implements iterative multi-judge debate:
22
23	```
24	Phase 0: Setup
25	mkdir -p .specs/reports
26	\|
27	Phase 0.5: Dispatch Meta-Judge
28	Meta-Judge (Opus)
29	\|
30	Evaluation Specification YAML
31	\|
32	Phase 1: Independent Analysis (3 judges in parallel)
33	+- Judge 1 -> {name}.1.md -+
34	Solution +- Judge 2 -> {name}.2.md -+-+
35	+- Judge 3 -> {name}.3.md -+ \|
36	\|
37	Phase 2: Debate Round (iterative) \|
38	Each judge reads others' reports \|
39	\| \|
40	Argue + Defend + Challenge \|
41	(grounded in eval specification) \|
42	\| \|
43	Revise if convinced --------------+
44	\| \|
45	Check consensus \|
46	+- Yes -> Final Report \|
47	+- No -> Next Round ---------+
48	```
49
50	## Process
51
52	### Setup: Create Reports Directory
53
54	Before starting evaluation, ensure the reports directory exists:
55
56	```bash
57	mkdir -p .specs/reports
58	```
59
60	Report naming convention: `.specs/reports/{solution-name}-{YYYY-MM-DD}.[1\|2\|3].md`
61
62	Where:
63	- `{solution-name}` - Derived from solution filename (e.g., `users-api` from `src/api/users.ts`)
64	- `{YYYY-MM-DD}` - Current date
65	- `[1\|2\|3]` - Judge number
66
67	### Phase 0.5: Dispatch Meta-Judge
68
69	Before independent analysis, dispatch a meta-judge agent to generate a tailored evaluation specification. The meta-judge runs ONCE and produces rubrics, checklists, and scoring criteria that ALL judges will use across ALL rounds.
70
71	Meta-judge prompt template:
72
73	```markdown
74	## Task
75
76	Generate an evaluation specification yaml for the following evaluation task. You will produce rubrics, checklists, and scoring criteria that multiple judge agents will use to evaluate the solution through independent analysis and multi-round debate.
77
78	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
79
80	## User Prompt
81	{task description - what the solution was supposed to accomplish}
82
83	## Context
84	{Any relevant context about the solution being evaluated}
85
86	## Artifact Type
87	{code \| documentation \| configuration \| etc.}
88
89	## Evaluation Mode
90	Multi-judge debate with consensus-seeking across rounds
91
92	## Instructions
93	Return only the final evaluation specification YAML in your response.
94	The specification should support both independent analysis and debate-based refinement.
95	```
96
97	Dispatch:
98
99	```
100	Use Task tool:
101	- description: "Meta-judge: generate evaluation specification for {solution-name}"
102	- prompt: {meta-judge prompt}
103	- model: opus
104	- subagent_type: "sadd:meta-judge"
105	```
106
107	Wait for the meta-judge to complete and extract the evaluation specification YAML from its output before proceeding to Phase 1.
108
109	### Phase 1: Independent Analysis
110
111	Launch 3 independent judge agents in parallel (Opus for rigor):
112
113	1. Each judge receives:
114	- Path to solution(s) being evaluated
115	- The meta-judge's evaluation specification YAML
116	- Task description
117	2. Each produces independent assessment saved to `.specs/reports/{solution-name}-{date}.[1\|2\|3].md`
118	3. Reports must include:
119	- Per-criterion scores with evidence
120	- Specific quotes/examples supporting ratings
121	- Overall weighted score
122	- Key strengths and weaknesses
123
124	Key principle: Independence in initial analysis prevents groupthink.
125
126	Prompt template for initial judges:
127
128	```markdown
129	You are Judge {N} evaluating a solution independently against an evaluation specification produced by the meta judge.
130
131	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
132
133	## Solution
134	{path to solution file(s)}
135
136	## Task Description
137	{what the solution was supposed to accomplish}
138
139	## Evaluation Specification
140
141	`

People who liked this also viewed

6 picks

Type