pi-skillopt

Name: pi-skillopt
Author: mitkox

80stars19forks

How to install pi-skillopt?

mitkox/skillopt/pi-skillopt

$npx -y skills add mitkox/SkillOpt --skill pi-skillopt

Installs into the current project.

Summary

Use when: running SkillOpt, training a skill, evaluating a skill, using the local mitko model on port 8000, working with the dotnetdebug benchmark, adding a benchmark, adding a backend, or optimizing agent instructions with SkillOpt in this repository.

Files · 1

View on GitHub

SKILL.md

1	# PI SkillOpt Operator
2
3	Use this skill when the user wants the agent to operate the SkillOpt repository: run experiments, evaluate skills, configure local models, inspect outputs, or extend the repo with new benchmarks/backends.
4
5	## Core repo facts
6
7	- SkillOpt optimizes a skill document / system prompt, not model weights.
8	- The repo supports a generic local OpenAI-compatible backend named `openai_compat`.
9	- The local example model config is `configs/dotnetdebug/local_mitko.yaml`.
10	- That config uses model `mitko` at `http://localhost:8000/v1` for both optimizer and target.
11	- The runnable sample benchmark is `dotnetdebug`.
12	- The sample dataset is `data/dotnetdebug/tasks.json`.
13	- The seed skill is `skillopt/envs/dotnetdebug/skills/initial.md`.
14	- Training entry point: `scripts/train.py`.
15	- Eval-only entry point: `scripts/eval_only.py`.
16
17	## Default workflow
18
19	1. Identify the user goal:
20	- run a SkillOpt training job
21	- evaluate an existing skill
22	- inspect outputs from a previous run
23	- add or modify a benchmark
24	- add or modify a backend
25	2. Confirm the benchmark, backend, target model, and desired output directory.
26	3. Prefer a small smoke test first before a larger run.
27	4. Use existing configs when possible instead of inventing new ones.
28	5. After a run, report:
29	- exact command used
30	- output directory
31	- best skill path
32	- headline metrics
33	- next recommended action
34
35	## Local model workflow
36
37	When the user asks to use a local model or mentions `mitko`, `localhost:8000`, or an OpenAI-compatible endpoint:
38
39	- Prefer `model.backend: openai_compat`.
40	- Set both `optimizer_backend` and `target_backend` to `openai_compat` unless the user explicitly wants a mixed setup.
41	- Use `configs/dotnetdebug/local_mitko.yaml` when the task is the built-in dotnet debugging example.
42	- Keep smoke tests small, for example:
43	- `train.num_epochs=1`
44	- `train.batch_size=2`
45	- `gradient.analyst_workers=1`
46	- `gradient.minibatch_size=2`
47	- `env.workers=1`
48	- `env.limit=2`
49
50	## Common commands
51
52	Activate the environment first if `.venv` exists:
53
54	```bash
55	source .venv/bin/activate
56	```
57
58	Small local training run:
59
60	```bash
61	python3 scripts/train.py \
62	--config configs/dotnetdebug/local_mitko.yaml \
63	--cfg-options \
64	train.num_epochs=1 \
65	train.batch_size=2 \
66	gradient.analyst_workers=1 \
67	gradient.minibatch_size=2 \
68	env.workers=1 \
69	env.limit=2 \
70	optimizer.learning_rate=2 \
71	env.out_root=outputs/dotnetdebug_smoke
72	```
73
74	Eval-only run:
75
76	```bash
77	python3 scripts/eval_only.py \
78	--config configs/dotnetdebug/local_mitko.yaml \
79	--skill outputs/dotnetdebug_smoke/best_skill.md \
80	--split test \
81	--cfg-options env.limit=2 env.workers=1 env.out_root=outputs/dotnetdebug_eval_smoke
82	```
83
84	## Extension workflow
85
86	When adding a new benchmark:
87
88	- Create files under `skillopt/envs/<benchmark>/`.
89	- Add config under `configs/<benchmark>/default.yaml`.
90	- Register the adapter in both:
91	- `scripts/train.py`
92	- `scripts/eval_only.py`
93	- Add or update docs when the new benchmark is user-facing.
94
95	When adding a new backend:
96
97	- Add the backend module under `skillopt/model/`.
98	- Update backend normalization and defaults in `skillopt/model/common.py`.
99	- Update allowed runtime backends in `skillopt/model/backend_config.py`.
100	- Update dispatch in `skillopt/model/__init__.py`.
101	- Expose config in `skillopt/config.py`, `configs/_base_/default.yaml`, and CLI entry points.
102
103	## Guardrails
104
105	- Do not describe SkillOpt as model fine-tuning.
106	- Do not start with a long expensive run if a smoke test can validate the setup first.
107	- Prefer repo-native paths and configs over ad hoc scripts.
108	- If editing benchmark/backend code, validate syntax/errors after changes.
109	- If a run fails, report the first concrete failure point and propose the smallest next fix.
110
111	## Response checklist
112
113	Before finishing, make sure the response includes:
114
115	- the chosen benchmark/config/backend
116	- the exact command(s) to run
117	- where outputs will be written
118	- what artifact to inspect next (`best_skill.md`, eval summary, predictions, patches, etc.)

People who liked this also viewed

6 picks

Type

pi-skillopt

80stars19forks

View on GitHub

How to install pi-skillopt?

mitkox/skillopt/pi-skillopt

$npx -y skills add mitkox/SkillOpt --skill pi-skillopt

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# PI SkillOpt Operator
2
3	Use this skill when the user wants the agent to operate the SkillOpt repository: run experiments, evaluate skills, configure local models, inspect outputs, or extend the repo with new benchmarks/backends.
4
5	## Core repo facts
6
7	- SkillOpt optimizes a skill document / system prompt, not model weights.
8	- The repo supports a generic local OpenAI-compatible backend named `openai_compat`.
9	- The local example model config is `configs/dotnetdebug/local_mitko.yaml`.
10	- That config uses model `mitko` at `http://localhost:8000/v1` for both optimizer and target.
11	- The runnable sample benchmark is `dotnetdebug`.
12	- The sample dataset is `data/dotnetdebug/tasks.json`.
13	- The seed skill is `skillopt/envs/dotnetdebug/skills/initial.md`.
14	- Training entry point: `scripts/train.py`.
15	- Eval-only entry point: `scripts/eval_only.py`.
16
17	## Default workflow
18
19	1. Identify the user goal:
20	- run a SkillOpt training job
21	- evaluate an existing skill
22	- inspect outputs from a previous run
23	- add or modify a benchmark
24	- add or modify a backend
25	2. Confirm the benchmark, backend, target model, and desired output directory.
26	3. Prefer a small smoke test first before a larger run.
27	4. Use existing configs when possible instead of inventing new ones.
28	5. After a run, report:
29	- exact command used
30	- output directory
31	- best skill path
32	- headline metrics
33	- next recommended action
34
35	## Local model workflow
36
37	When the user asks to use a local model or mentions `mitko`, `localhost:8000`, or an OpenAI-compatible endpoint:
38
39	- Prefer `model.backend: openai_compat`.
40	- Set both `optimizer_backend` and `target_backend` to `openai_compat` unless the user explicitly wants a mixed setup.
41	- Use `configs/dotnetdebug/local_mitko.yaml` when the task is the built-in dotnet debugging example.
42	- Keep smoke tests small, for example:
43	- `train.num_epochs=1`
44	- `train.batch_size=2`
45	- `gradient.analyst_workers=1`
46	- `gradient.minibatch_size=2`
47	- `env.workers=1`
48	- `env.limit=2`
49
50	## Common commands
51
52	Activate the environment first if `.venv` exists:
53
54	```bash
55	source .venv/bin/activate
56	```
57
58	Small local training run:
59
60	```bash
61	python3 scripts/train.py \
62	--config configs/dotnetdebug/local_mitko.yaml \
63	--cfg-options \
64	train.num_epochs=1 \
65	train.batch_size=2 \
66	gradient.analyst_workers=1 \
67	gradient.minibatch_size=2 \
68	env.workers=1 \
69	env.limit=2 \
70	optimizer.learning_rate=2 \
71	env.out_root=outputs/dotnetdebug_smoke
72	```
73
74	Eval-only run:
75
76	```bash
77	python3 scripts/eval_only.py \
78	--config configs/dotnetdebug/local_mitko.yaml \
79	--skill outputs/dotnetdebug_smoke/best_skill.md \
80	--split test \
81	--cfg-options env.limit=2 env.workers=1 env.out_root=outputs/dotnetdebug_eval_smoke
82	```
83
84	## Extension workflow
85
86	When adding a new benchmark:
87
88	- Create files under `skillopt/envs/<benchmark>/`.
89	- Add config under `configs/<benchmark>/default.yaml`.
90	- Register the adapter in both:
91	- `scripts/train.py`
92	- `scripts/eval_only.py`
93	- Add or update docs when the new benchmark is user-facing.
94
95	When adding a new backend:
96
97	- Add the backend module under `skillopt/model/`.
98	- Update backend normalization and defaults in `skillopt/model/common.py`.
99	- Update allowed runtime backends in `skillopt/model/backend_config.py`.
100	- Update dispatch in `skillopt/model/__init__.py`.
101	- Expose config in `skillopt/config.py`, `configs/_base_/default.yaml`, and CLI entry points.
102
103	## Guardrails
104
105	- Do not describe SkillOpt as model fine-tuning.
106	- Do not start with a long expensive run if a smoke test can validate the setup first.
107	- Prefer repo-native paths and configs over ad hoc scripts.
108	- If editing benchmark/backend code, validate syntax/errors after changes.
109	- If a run fails, report the first concrete failure point and propose the smallest next fix.
110
111	## Response checklist
112
113	Before finishing, make sure the response includes:
114
115	- the chosen benchmark/config/backend
116	- the exact command(s) to run
117	- where outputs will be written
118	- what artifact to inspect next (`best_skill.md`, eval summary, predictions, patches, etc.)

People who liked this also viewed

6 picks

Type