skill-forge-evolve

Name: skill-forge-evolve
Author: AgriciDaniel

78stars33forks

How to install skill-forge-evolve?

agricidaniel/skill-forge/skill-forge-evolve

$npx -y skills add AgriciDaniel/skill-forge --skill skill-forge-evolve

Installs into the current project.

Summary

Improve and iterate on existing Claude Code skills based on usage feedback, test results, or changing requirements. Handles under/over-triggering fixes, instruction refinement, new sub-skill addition, and architecture evolution. Use when user says "improve skill", "fix skill", "s

Files · 1

View on GitHub

SKILL.md

1	# Skill Evolution & Improvement
2
3	## Process
4
5	### Step 1: Diagnose the Issue
6
7	Ask the user or analyze logs to identify the problem category:
8
9	Category A: Triggering Issues
10	- Under-triggering: Skill doesn't activate when it should
11	- Over-triggering: Skill activates when it shouldn't
12	- Mis-triggering: Wrong sub-skill activates
13
14	Category B: Execution Issues
15	- Incomplete workflows: Skill stops before finishing
16	- Incorrect output: Results don't match expectations
17	- Missing error handling: Failures not handled gracefully
18	- Performance: Too slow or too many token used
19
20	Category C: Architecture Issues
21	- Missing capability: New use case not covered
22	- Scale issues: Skill too large, needs decomposition
23	- Cross-reference problems: Links to non-existent files
24
25	Category D: Quality Issues
26	- Inconsistent results: Different outputs for same input
27	- Vague instructions: Claude interprets differently each time
28	- Missing examples: No concrete guidance
29
30	### Step 2: Apply Fix by Category
31
32	#### Fix: Under-Triggering
33
34	1. Read current description
35	2. Identify missing trigger phrases
36	3. Add domain keywords and paraphrases
37	4. Add file type mentions if relevant
38	5. Test with 5 queries that should now trigger
39
40	Common causes:
41	- Description too generic
42	- Missing common paraphrases
43	- Technical jargon without lay terms
44
45	Fix template:
46	```yaml
47	# Before (under-triggers)
48	description: Analyzes code quality
49
50	# After (specific triggers)
51	description: >
52	Static code analysis and quality assessment. Checks code style,
53	complexity, security vulnerabilities, and test coverage. Use when
54	user says "code review", "code quality", "lint", "static analysis",
55	"code smell", "code audit", or "check my code".
56	```
57
58	#### Fix: Over-Triggering
59
60	1. Read current description
61	2. Identify why unrelated queries trigger it
62	3. Add negative triggers ("Do NOT use for...")
63	4. Make description more specific
64	5. Test with 5 queries that should NOT trigger
65
66	Fix template:
67	```yaml
68	# Before (over-triggers)
69	description: Processes documents for review
70
71	# After (specific + negative triggers)
72	description: >
73	Processes PDF legal documents for contract clause extraction and
74	compliance review. Use for legal contracts, NDAs, terms of service.
75	Do NOT use for general document editing, formatting, or non-legal PDFs.
76	```
77
78	#### Fix: Execution Issues
79
80	1. Identify the failing step in the workflow
81	2. Add explicit validation gates between steps
82	3. Add error handling with clear recovery instructions
83	4. Add "If X fails, then Y" fallback paths
84	5. Consider adding a script for fragile operations
85
86	#### Fix: Quality Issues
87
88	1. Replace vague instructions with specific ones
89	2. Add concrete examples of expected input/output
90	3. Add explicit "do this, not that" comparisons
91	4. Add quality check steps before final output
92	5. Consider adding a validation script
93
94	### Step 3: Iteration Workspace Protocol
95
96	Use structured workspaces to track improvements across iterations:
97
98	```
99	eval-workspace/
100	iteration-1/ # First version
101	eval-0/with_skill/ # Eval results
102	eval-0/baseline/
103	benchmark.json # Aggregated metrics
104	benchmark.md # Human-readable report
105	feedback.json # User feedback
106	iteration-2/ # After first improvement
107	eval-0/with_skill/
108	eval-0/baseline/
109	benchmark.json
110	benchmark.md
111	feedback.json
112	```
113
114	The iteration loop:
115	1. Apply the fix to the skill
116	2. Run `/skill-forge eval <path>` into `iteration-<N+1>/`
117	3. Run `/skill-forge benchmark <path>` with `--previous iteration-<N>/`
118	4. Review benchmark comparison for regressions
119	5. Collect user feedback into `feedback.json`
120	6. Read feedback and iterate (back to Step 2)
121
122	Stop iterating when:
123	- User says they're happy
124	- All feedback is empty (everything looks good)
125	- Benchmark shows no meaningful progress between iterations
126	- Pass rate meets the defined thresholds
127
128	### Step 3b: Self-Annealing Loop
129
130	For quick fixes without full eval pipeline:
131
132	```
133	1. Apply the fix
134	2. Test with the original failing case
135	3. Test with 3 other cases (regression check)
136	4. If fix works:
137	-> Update the directive/SKILL.md
138	-> Document the learning in references or SKILL.md
139	5. If fix fails:
140	-> Diagnose why
141	-> Try alternative approach
142	-> Repeat
143	```
144
145	### Step 3c: Description Optimization Loop
146
147	For triggering issues (Category A), use the automated optimization loop:
148
149	1. Generate trigger eval set: `python scripts/generate_eval_set.py <path>`
150	2. Review and refine the eval set with the user
151	3. Run optimization: `python scripts/optimize_description.py <path> --eval-set

People who liked this also viewed

6 picks

Type

skill-forge-evolve

78stars33forks

View on GitHub

How to install skill-forge-evolve?

agricidaniel/skill-forge/skill-forge-evolve

$npx -y skills add AgriciDaniel/skill-forge --skill skill-forge-evolve

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Skill Evolution & Improvement
2
3	## Process
4
5	### Step 1: Diagnose the Issue
6
7	Ask the user or analyze logs to identify the problem category:
8
9	Category A: Triggering Issues
10	- Under-triggering: Skill doesn't activate when it should
11	- Over-triggering: Skill activates when it shouldn't
12	- Mis-triggering: Wrong sub-skill activates
13
14	Category B: Execution Issues
15	- Incomplete workflows: Skill stops before finishing
16	- Incorrect output: Results don't match expectations
17	- Missing error handling: Failures not handled gracefully
18	- Performance: Too slow or too many token used
19
20	Category C: Architecture Issues
21	- Missing capability: New use case not covered
22	- Scale issues: Skill too large, needs decomposition
23	- Cross-reference problems: Links to non-existent files
24
25	Category D: Quality Issues
26	- Inconsistent results: Different outputs for same input
27	- Vague instructions: Claude interprets differently each time
28	- Missing examples: No concrete guidance
29
30	### Step 2: Apply Fix by Category
31
32	#### Fix: Under-Triggering
33
34	1. Read current description
35	2. Identify missing trigger phrases
36	3. Add domain keywords and paraphrases
37	4. Add file type mentions if relevant
38	5. Test with 5 queries that should now trigger
39
40	Common causes:
41	- Description too generic
42	- Missing common paraphrases
43	- Technical jargon without lay terms
44
45	Fix template:
46	```yaml
47	# Before (under-triggers)
48	description: Analyzes code quality
49
50	# After (specific triggers)
51	description: >
52	Static code analysis and quality assessment. Checks code style,
53	complexity, security vulnerabilities, and test coverage. Use when
54	user says "code review", "code quality", "lint", "static analysis",
55	"code smell", "code audit", or "check my code".
56	```
57
58	#### Fix: Over-Triggering
59
60	1. Read current description
61	2. Identify why unrelated queries trigger it
62	3. Add negative triggers ("Do NOT use for...")
63	4. Make description more specific
64	5. Test with 5 queries that should NOT trigger
65
66	Fix template:
67	```yaml
68	# Before (over-triggers)
69	description: Processes documents for review
70
71	# After (specific + negative triggers)
72	description: >
73	Processes PDF legal documents for contract clause extraction and
74	compliance review. Use for legal contracts, NDAs, terms of service.
75	Do NOT use for general document editing, formatting, or non-legal PDFs.
76	```
77
78	#### Fix: Execution Issues
79
80	1. Identify the failing step in the workflow
81	2. Add explicit validation gates between steps
82	3. Add error handling with clear recovery instructions
83	4. Add "If X fails, then Y" fallback paths
84	5. Consider adding a script for fragile operations
85
86	#### Fix: Quality Issues
87
88	1. Replace vague instructions with specific ones
89	2. Add concrete examples of expected input/output
90	3. Add explicit "do this, not that" comparisons
91	4. Add quality check steps before final output
92	5. Consider adding a validation script
93
94	### Step 3: Iteration Workspace Protocol
95
96	Use structured workspaces to track improvements across iterations:
97
98	```
99	eval-workspace/
100	iteration-1/ # First version
101	eval-0/with_skill/ # Eval results
102	eval-0/baseline/
103	benchmark.json # Aggregated metrics
104	benchmark.md # Human-readable report
105	feedback.json # User feedback
106	iteration-2/ # After first improvement
107	eval-0/with_skill/
108	eval-0/baseline/
109	benchmark.json
110	benchmark.md
111	feedback.json
112	```
113
114	The iteration loop:
115	1. Apply the fix to the skill
116	2. Run `/skill-forge eval <path>` into `iteration-<N+1>/`
117	3. Run `/skill-forge benchmark <path>` with `--previous iteration-<N>/`
118	4. Review benchmark comparison for regressions
119	5. Collect user feedback into `feedback.json`
120	6. Read feedback and iterate (back to Step 2)
121
122	Stop iterating when:
123	- User says they're happy
124	- All feedback is empty (everything looks good)
125	- Benchmark shows no meaningful progress between iterations
126	- Pass rate meets the defined thresholds
127
128	### Step 3b: Self-Annealing Loop
129
130	For quick fixes without full eval pipeline:
131
132	```
133	1. Apply the fix
134	2. Test with the original failing case
135	3. Test with 3 other cases (regression check)
136	4. If fix works:
137	-> Update the directive/SKILL.md
138	-> Document the learning in references or SKILL.md
139	5. If fix fails:
140	-> Diagnose why
141	-> Try alternative approach
142	-> Repeat
143	```
144
145	### Step 3c: Description Optimization Loop
146
147	For triggering issues (Category A), use the automated optimization loop:
148
149	1. Generate trigger eval set: `python scripts/generate_eval_set.py <path>`
150	2. Review and refine the eval set with the user
151	3. Run optimization: `python scripts/optimize_description.py <path> --eval-set

People who liked this also viewed

6 picks

Type