judge

Name: judge
Author: NeoLabHQ

856installs132forks

How to install judge?

neolabhq/context-engineering-kit/judge

$npx -y skills add NeoLabHQ/context-engineering-kit --skill judge

Installs into the current project.

Summary

Launch a meta-judge then a judge sub-agent to evaluate results produced in the current conversation

Files · 1

View on GitHub

SKILL.md

1	# Judge Command
2
3	<task>
4	You are a coordinator launching a two-phase evaluation pipeline to assess work produced earlier in this conversation. First, a meta-judge generates tailored evaluation criteria. Then, a judge sub-agent applies those criteria with isolated context, structured scoring, and evidence-based feedback. The evaluation is report-only - findings are presented without automatic changes.
5	</task>
6
7	<context>
8	This command implements the meta-judge -> LLM-as-Judge pattern with context isolation:
9	- Structured Evaluation: Meta-judge produces tailored rubrics, checklists, and scoring criteria before judging
10	- Context Isolation: Judge operates with fresh context, preventing confirmation bias from accumulated session state
11	- Evidence-Based: Every score requires specific citations from the work (file locations, line numbers)
12	- Multi-Dimensional Rubric: Generated by meta-judge to match the specific artifact type and evaluation focus
13	- Self-Verification: Dynamic verification questions with documented adjustments
14	</context>
15
16	## Your Workflow
17
18	### Phase 1: Context Extraction
19
20	Before launching the evaluation pipeline, identify what needs evaluation:
21
22	1. Identify the work to evaluate:
23	- Review conversation history for completed work
24	- If arguments provided: Use them to focus on specific aspects
25	- If unclear: Ask user "What work should I evaluate? (code changes, analysis, documentation, etc.)"
26
27	2. Extract evaluation context:
28	- Original task or request that prompted the work
29	- The actual output/result produced
30	- Files created or modified (with brief descriptions)
31	- Any constraints, requirements, or acceptance criteria mentioned
32	- Artifact type (code, documentation, configuration, etc.)
33
34	3. Provide scope for user:
35
36	```
37	Evaluation Scope:
38	- Original request: [summary]
39	- Work produced: [description]
40	- Files involved: [list]
41	- Artifact type: [code \| documentation \| configuration \| etc.]
42	- Evaluation focus: [from arguments or "general quality"]
43
44	Launching meta-judge to generate evaluation criteria...
45	```
46
47	IMPORTANT: Pass only the extracted context to the sub-agents - not the entire conversation. This prevents context pollution and enables focused assessment.
48
49	### Phase 2: Dispatch Meta-Judge
50
51	Launch a meta-judge agent to generate an evaluation specification tailored to the specific work being evaluated. The meta-judge will return an evaluation specification YAML containing rubrics, checklists, and scoring criteria.
52
53	Meta-Judge Prompt:
54
55	```markdown
56	## Task
57
58	Generate an evaluation specification yaml for the following evaluation task. You will produce rubrics, checklists, and scoring criteria that a judge agent will use to evaluate the work.
59
60	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
61
62	## User Prompt
63	{Original task or request that prompted the work}
64
65	## Context
66	{Any relevant context about the work being evaluated}
67	{Evaluation focus from arguments, or "General quality assessment"}
68
69	## Artifact Type
70	{code \| documentation \| configuration \| etc.}
71
72	## Instructions
73	Return only the final evaluation specification YAML in your response.
74	```
75
76	Dispatch:
77
78	```
79	Use Task tool:
80	- description: "Meta-judge: Generate evaluation criteria for {brief work summary}"
81	- prompt: {meta-judge prompt}
82	- model: opus
83	- subagent_type: "sadd:meta-judge"
84	```
85
86	Wait for the meta-judge to complete before proceeding to Phase 3.
87
88	### Phase 3: Dispatch Judge Agent
89
90	After the meta-judge completes, extract its evaluation specification YAML and dispatch the judge agent with both the work context and the specification.
91
92	CRITICAL: Provide to the judge the EXACT meta-judge evaluation specification YAML. Do not skip, add, modify, shorten, or summarize any text in it!
93
94	Judge Agent Prompt:
95
96	```markdown
97	You are an Expert Judge evaluating the quality of work against an evaluation specification produced by the meta judge.
98
99	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
100
101	## Work Under Evaluation
102
103	[ORIGINAL TASK]
104	{paste the original request/task}
105	[/ORIGINAL TASK]
106
107	[WORK OUTPUT]
108	{summary of what was created/modified}
109	[/WORK OUTPUT]
110
111	[FILES INVOLVED]
112	{list of files with brief descriptions}
113	[/FILES INVOLVED]
114
115	## Evaluation Specification
116
117	```yaml
118	{meta-judge's evaluation specification YAML}
119	```
120
121	## Instructions
122
123	Follow your full judge process as defined in your agent instructions!
124
125	CRITICAL: You must reply with this exact structured evaluation report format in YAML at the START of your response!
126	```
127
128	CRITICAL: NEVER provide score threshold to judges in any format. Judge MUST not know what threshold for score is, in order to not be biased!!!
129
130	Dispatch:
131
132	```
133	Use Task tool:
134	- description: "Judge: Evaluate {brief work summary}"
135	- prompt: {judge prompt with exact meta-judge specification YAML}
136	- mo

People who liked this also viewed

6 picks

Type

tddSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026501k181k
test-driven-developmentUse when implementing any feature or bugfix, before writing implementation codeSkillsJul 2026175k259k
qaSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026160k181k
verification-before-completionUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidenceSkillsJul 2026153k259k
webapp-testingToolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser lSkillsJul 2026120k163k
playwright-best-practicesUse when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, teSkillsJul 202664k338

judge

856installs132forks

View on GitHub

How to install judge?

neolabhq/context-engineering-kit/judge

$npx -y skills add NeoLabHQ/context-engineering-kit --skill judge

Installs into the current project.

Summary

Launch a meta-judge then a judge sub-agent to evaluate results produced in the current conversation

Files · 1

View on GitHub

SKILL.md

1	# Judge Command
2
3	<task>
4	You are a coordinator launching a two-phase evaluation pipeline to assess work produced earlier in this conversation. First, a meta-judge generates tailored evaluation criteria. Then, a judge sub-agent applies those criteria with isolated context, structured scoring, and evidence-based feedback. The evaluation is report-only - findings are presented without automatic changes.
5	</task>
6
7	<context>
8	This command implements the meta-judge -> LLM-as-Judge pattern with context isolation:
9	- Structured Evaluation: Meta-judge produces tailored rubrics, checklists, and scoring criteria before judging
10	- Context Isolation: Judge operates with fresh context, preventing confirmation bias from accumulated session state
11	- Evidence-Based: Every score requires specific citations from the work (file locations, line numbers)
12	- Multi-Dimensional Rubric: Generated by meta-judge to match the specific artifact type and evaluation focus
13	- Self-Verification: Dynamic verification questions with documented adjustments
14	</context>
15
16	## Your Workflow
17
18	### Phase 1: Context Extraction
19
20	Before launching the evaluation pipeline, identify what needs evaluation:
21
22	1. Identify the work to evaluate:
23	- Review conversation history for completed work
24	- If arguments provided: Use them to focus on specific aspects
25	- If unclear: Ask user "What work should I evaluate? (code changes, analysis, documentation, etc.)"
26
27	2. Extract evaluation context:
28	- Original task or request that prompted the work
29	- The actual output/result produced
30	- Files created or modified (with brief descriptions)
31	- Any constraints, requirements, or acceptance criteria mentioned
32	- Artifact type (code, documentation, configuration, etc.)
33
34	3. Provide scope for user:
35
36	```
37	Evaluation Scope:
38	- Original request: [summary]
39	- Work produced: [description]
40	- Files involved: [list]
41	- Artifact type: [code \| documentation \| configuration \| etc.]
42	- Evaluation focus: [from arguments or "general quality"]
43
44	Launching meta-judge to generate evaluation criteria...
45	```
46
47	IMPORTANT: Pass only the extracted context to the sub-agents - not the entire conversation. This prevents context pollution and enables focused assessment.
48
49	### Phase 2: Dispatch Meta-Judge
50
51	Launch a meta-judge agent to generate an evaluation specification tailored to the specific work being evaluated. The meta-judge will return an evaluation specification YAML containing rubrics, checklists, and scoring criteria.
52
53	Meta-Judge Prompt:
54
55	```markdown
56	## Task
57
58	Generate an evaluation specification yaml for the following evaluation task. You will produce rubrics, checklists, and scoring criteria that a judge agent will use to evaluate the work.
59
60	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
61
62	## User Prompt
63	{Original task or request that prompted the work}
64
65	## Context
66	{Any relevant context about the work being evaluated}
67	{Evaluation focus from arguments, or "General quality assessment"}
68
69	## Artifact Type
70	{code \| documentation \| configuration \| etc.}
71
72	## Instructions
73	Return only the final evaluation specification YAML in your response.
74	```
75
76	Dispatch:
77
78	```
79	Use Task tool:
80	- description: "Meta-judge: Generate evaluation criteria for {brief work summary}"
81	- prompt: {meta-judge prompt}
82	- model: opus
83	- subagent_type: "sadd:meta-judge"
84	```
85
86	Wait for the meta-judge to complete before proceeding to Phase 3.
87
88	### Phase 3: Dispatch Judge Agent
89
90	After the meta-judge completes, extract its evaluation specification YAML and dispatch the judge agent with both the work context and the specification.
91
92	CRITICAL: Provide to the judge the EXACT meta-judge evaluation specification YAML. Do not skip, add, modify, shorten, or summarize any text in it!
93
94	Judge Agent Prompt:
95
96	```markdown
97	You are an Expert Judge evaluating the quality of work against an evaluation specification produced by the meta judge.
98
99	CLAUDE_PLUGIN_ROOT=`${CLAUDE_PLUGIN_ROOT}`
100
101	## Work Under Evaluation
102
103	[ORIGINAL TASK]
104	{paste the original request/task}
105	[/ORIGINAL TASK]
106
107	[WORK OUTPUT]
108	{summary of what was created/modified}
109	[/WORK OUTPUT]
110
111	[FILES INVOLVED]
112	{list of files with brief descriptions}
113	[/FILES INVOLVED]
114
115	## Evaluation Specification
116
117	```yaml
118	{meta-judge's evaluation specification YAML}
119	```
120
121	## Instructions
122
123	Follow your full judge process as defined in your agent instructions!
124
125	CRITICAL: You must reply with this exact structured evaluation report format in YAML at the START of your response!
126	```
127
128	CRITICAL: NEVER provide score threshold to judges in any format. Judge MUST not know what threshold for score is, in order to not be biased!!!
129
130	Dispatch:
131
132	```
133	Use Task tool:
134	- description: "Judge: Evaluate {brief work summary}"
135	- prompt: {judge prompt with exact meta-judge specification YAML}
136	- mo

People who liked this also viewed

6 picks

Type