skill-judge

Name: skill-judge
Author: shareAI-lab

304stars52forks

How to install skill-judge?

shareai-lab/shareai-skills/skill-judge

$npx -y skills add shareAI-lab/shareAI-skills --skill skill-judge

Installs into the current project.

Summary

Evaluate Agent Skill design quality against official specifications and best practices. Use when reviewing, auditing, or improving SKILL.md files and skill packages. Provides multi-dimensional scoring and actionable improvement suggestions.

Files · 1

View on GitHub

SKILL.md

1	# Skill Judge
2
3	Evaluate Agent Skills against official specifications and patterns derived from 17+ official examples.
4
5	---
6
7	## Core Philosophy
8
9	### What is a Skill?
10
11	A Skill is NOT a tutorial. A Skill is a knowledge externalization mechanism.
12
13	Traditional AI knowledge is locked in model parameters. To teach new capabilities:
14	```
15	Traditional: Collect data → GPU cluster → Train → Deploy new version
16	Cost: $10,000 - $1,000,000+
17	Timeline: Weeks to months
18	```
19
20	Skills change this:
21	```
22	Skill: Edit SKILL.md → Save → Takes effect on next invocation
23	Cost: $0
24	Timeline: Instant
25	```
26
27	This is the paradigm shift from "training AI" to "educating AI" — like a hot-swappable LoRA adapter that requires no training. You edit a Markdown file in natural language, and the model's behavior changes.
28
29	### The Core Formula
30
31	> Good Skill = Expert-only Knowledge − What Claude Already Knows
32
33	A Skill's value is measured by its knowledge delta — the gap between what it provides and what the model already knows.
34
35	- Expert-only knowledge: Decision trees, trade-offs, edge cases, anti-patterns, domain-specific thinking frameworks — things that take years of experience to accumulate
36	- What Claude already knows: Basic concepts, standard library usage, common programming patterns, general best practices
37
38	When a Skill explains "what is PDF" or "how to write a for-loop", it's compressing knowledge Claude already has. This is token waste — context window is a public resource shared with system prompts, conversation history, other Skills, and user requests.
39
40	### Tool vs Skill
41
42	\| Concept \| Essence \| Function \| Example \|
43	\|---------\|---------\|----------\|---------\|
44	\| Tool \| What model CAN do \| Execute actions \| bash, read_file, write_file, WebSearch \|
45	\| Skill \| What model KNOWS how to do \| Guide decisions \| PDF processing, MCP building, frontend design \|
46
47	Tools define capability boundaries — without bash tool, model can't execute commands.
48	Skills inject knowledge — without frontend-design Skill, model produces generic UI.
49
50	The equation:
51	```
52	General Agent + Excellent Skill = Domain Expert Agent
53	```
54
55	Same Claude model, different Skills loaded, becomes different experts.
56
57	### Three Types of Knowledge in Skills
58
59	When evaluating, categorize each section:
60
61	\| Type \| Definition \| Treatment \|
62	\|------\|------------\|-----------\|
63	\| Expert \| Claude genuinely doesn't know this \| Must keep — this is the Skill's value \|
64	\| Activation \| Claude knows but may not think of \| Keep if brief — serves as reminder \|
65	\| Redundant \| Claude definitely knows this \| Should delete — wastes tokens \|
66
67	The art of Skill design is maximizing Expert content, using Activation sparingly, and eliminating Redundant ruthlessly.
68
69	---
70
71	## Evaluation Dimensions (120 points total)
72
73	### D1: Knowledge Delta (20 points) — THE CORE DIMENSION
74
75	The most important dimension. Does the Skill add genuine expert knowledge?
76
77	\| Score \| Criteria \|
78	\|-------\|----------\|
79	\| 0-5 \| Explains basics Claude knows (what is X, how to write code, standard library tutorials) \|
80	\| 6-10 \| Mixed: some expert knowledge diluted by obvious content \|
81	\| 11-15 \| Mostly expert knowledge with minimal redundancy \|
82	\| 16-20 \| Pure knowledge delta — every paragraph earns its tokens \|
83
84	Red flags (instant score ≤5):
85	- "What is [basic concept]" sections
86	- Step-by-step tutorials for standard operations
87	- Explaining how to use common libraries
88	- Generic best practices ("write clean code", "handle errors")
89	- Definitions of industry-standard terms
90
91	Green flags (indicators of high knowledge delta):
92	- Decision trees for non-obvious choices ("when X fails, try Y because Z")
93	- Trade-offs only an expert would know ("A is faster but B handles edge case C")
94	- Edge cases from real-world experience
95	- "NEVER do X because [non-obvious reason]"
96	- Domain-specific thinking frameworks
97
98	Evaluation questions:
99	1. For each section, ask: "Does Claude already know this?"
100	2. If explaining something, ask: "Is this explaining TO Claude or FOR Claude?"
101	3. Count paragraphs that are Expert vs Activation vs Redundant
102
103	---
104
105	### D2: Mindset + Appropriate Procedures (15 points)
106
107	Does the Skill transfer expert thinking patterns along with necessary domain-specific procedures?
108
109	The difference between experts and novices isn't "knowing how to operate" — it's "how to think about the problem." But thinking patterns alone aren't enough when Claude lacks domain-specific procedural knowledge.
110
111	Key distinction:
112	\| Type \| Example \| Value \|
113	\|------\|---------\|-------\|
114	\| Thinking patterns \| "Before designing, ask: What makes this memorable?" \| High — shapes decision-making \|
115	\| Domain-specific procedures \| "OOXML workflow: unpack → edit XML →

People who liked this also viewed

6 picks

Type

skill-judge

304stars52forks

View on GitHub

How to install skill-judge?

shareai-lab/shareai-skills/skill-judge

$npx -y skills add shareAI-lab/shareAI-skills --skill skill-judge

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Skill Judge
2
3	Evaluate Agent Skills against official specifications and patterns derived from 17+ official examples.
4
5	---
6
7	## Core Philosophy
8
9	### What is a Skill?
10
11	A Skill is NOT a tutorial. A Skill is a knowledge externalization mechanism.
12
13	Traditional AI knowledge is locked in model parameters. To teach new capabilities:
14	```
15	Traditional: Collect data → GPU cluster → Train → Deploy new version
16	Cost: $10,000 - $1,000,000+
17	Timeline: Weeks to months
18	```
19
20	Skills change this:
21	```
22	Skill: Edit SKILL.md → Save → Takes effect on next invocation
23	Cost: $0
24	Timeline: Instant
25	```
26
27	This is the paradigm shift from "training AI" to "educating AI" — like a hot-swappable LoRA adapter that requires no training. You edit a Markdown file in natural language, and the model's behavior changes.
28
29	### The Core Formula
30
31	> Good Skill = Expert-only Knowledge − What Claude Already Knows
32
33	A Skill's value is measured by its knowledge delta — the gap between what it provides and what the model already knows.
34
35	- Expert-only knowledge: Decision trees, trade-offs, edge cases, anti-patterns, domain-specific thinking frameworks — things that take years of experience to accumulate
36	- What Claude already knows: Basic concepts, standard library usage, common programming patterns, general best practices
37
38	When a Skill explains "what is PDF" or "how to write a for-loop", it's compressing knowledge Claude already has. This is token waste — context window is a public resource shared with system prompts, conversation history, other Skills, and user requests.
39
40	### Tool vs Skill
41
42	\| Concept \| Essence \| Function \| Example \|
43	\|---------\|---------\|----------\|---------\|
44	\| Tool \| What model CAN do \| Execute actions \| bash, read_file, write_file, WebSearch \|
45	\| Skill \| What model KNOWS how to do \| Guide decisions \| PDF processing, MCP building, frontend design \|
46
47	Tools define capability boundaries — without bash tool, model can't execute commands.
48	Skills inject knowledge — without frontend-design Skill, model produces generic UI.
49
50	The equation:
51	```
52	General Agent + Excellent Skill = Domain Expert Agent
53	```
54
55	Same Claude model, different Skills loaded, becomes different experts.
56
57	### Three Types of Knowledge in Skills
58
59	When evaluating, categorize each section:
60
61	\| Type \| Definition \| Treatment \|
62	\|------\|------------\|-----------\|
63	\| Expert \| Claude genuinely doesn't know this \| Must keep — this is the Skill's value \|
64	\| Activation \| Claude knows but may not think of \| Keep if brief — serves as reminder \|
65	\| Redundant \| Claude definitely knows this \| Should delete — wastes tokens \|
66
67	The art of Skill design is maximizing Expert content, using Activation sparingly, and eliminating Redundant ruthlessly.
68
69	---
70
71	## Evaluation Dimensions (120 points total)
72
73	### D1: Knowledge Delta (20 points) — THE CORE DIMENSION
74
75	The most important dimension. Does the Skill add genuine expert knowledge?
76
77	\| Score \| Criteria \|
78	\|-------\|----------\|
79	\| 0-5 \| Explains basics Claude knows (what is X, how to write code, standard library tutorials) \|
80	\| 6-10 \| Mixed: some expert knowledge diluted by obvious content \|
81	\| 11-15 \| Mostly expert knowledge with minimal redundancy \|
82	\| 16-20 \| Pure knowledge delta — every paragraph earns its tokens \|
83
84	Red flags (instant score ≤5):
85	- "What is [basic concept]" sections
86	- Step-by-step tutorials for standard operations
87	- Explaining how to use common libraries
88	- Generic best practices ("write clean code", "handle errors")
89	- Definitions of industry-standard terms
90
91	Green flags (indicators of high knowledge delta):
92	- Decision trees for non-obvious choices ("when X fails, try Y because Z")
93	- Trade-offs only an expert would know ("A is faster but B handles edge case C")
94	- Edge cases from real-world experience
95	- "NEVER do X because [non-obvious reason]"
96	- Domain-specific thinking frameworks
97
98	Evaluation questions:
99	1. For each section, ask: "Does Claude already know this?"
100	2. If explaining something, ask: "Is this explaining TO Claude or FOR Claude?"
101	3. Count paragraphs that are Expert vs Activation vs Redundant
102
103	---
104
105	### D2: Mindset + Appropriate Procedures (15 points)
106
107	Does the Skill transfer expert thinking patterns along with necessary domain-specific procedures?
108
109	The difference between experts and novices isn't "knowing how to operate" — it's "how to think about the problem." But thinking patterns alone aren't enough when Claude lacks domain-specific procedural knowledge.
110
111	Key distinction:
112	\| Type \| Example \| Value \|
113	\|------\|---------\|-------\|
114	\| Thinking patterns \| "Before designing, ask: What makes this memorable?" \| High — shapes decision-making \|
115	\| Domain-specific procedures \| "OOXML workflow: unpack → edit XML →

People who liked this also viewed

6 picks

Type