run-validation-until-no-failure

Name: run-validation-until-no-failure
Author: HLND2T

58stars10forks

How to install run-validation-until-no-failure?

hlnd2t/cs2_vibesignatures/run-validation-until-no-failure

$npx -y skills add HLND2T/CS2_VibeSignatures --skill run-validation-until-no-failure

Installs into the current project.

Summary

Use when the user wants to run the IDA validation pipeline (uv run ida_analyze_bin.py -debug) repeatedly until it reports zero failures. For each failing skill, diagnose the failure, ask the user how to proceed, and offer several viable solutions. Only when the user explicitly

Files · 1

View on GitHub

SKILL.md

1	# Run Validation Until No Failure
2
3	Loop the analysis/validation pipeline until it reports `Failed: 0`. Each iteration: run
4	`ida_analyze_bin.py -debug`; if a skill failed (the pipeline is fail-fast, so it's exactly one),
5	diagnose why, present several concrete resolution options, and ask the user which option to take.
6	Do not disable or record the skill before the user answers. If the user explicitly answers that it
7	cannot be solved now and asks to temporarily disable the skill, append the failure to
8	`docs/ida_validation_failure-<GAMEVER>.md`, comment that skill out of `configs/<GAMEVER>.yaml`, and
9	run the `unit` and `repository-contract` unittest suites before resuming validation. If the config
10	dependency test reports consumers whose `expected_input` can no longer be produced, apply the same
11	ask-before-action rule to each dependent skill. Environment / infrastructure failures are not
12	skill bugs — STOP and surface those instead of quarantining a skill.
13
14	Resolve `GAMEVER` from the user's explicit request or `CS2VIBE_GAMEVER`; use only
15	`configs/$GAMEVER.yaml` for the validation run and all quarantine edits.
16
17	## When to Use
18
19	- The user asks to "run validation until it passes / until no failures", "get validation green", or
20	"disable the failing skills and keep going".
21	- After bumping to a new game version or landing many new skills, to triage a wave of failures — each
22	gets quarantined with its reason for later fixing.
23
24	## Why the loop works (pipeline facts)
25
26	- `uv run ida_analyze_bin.py -debug` ends with a summary and exits non-zero when `Failed > 0`:
27	```
28	============================================================
29	Summary
30	============================================================
31	Successful: N
32	Failed: N
33	Skipped: N
34	```
35	- It is fully fail-fast: the first failing skill aborts the remaining skills, the remaining
36	platforms, and the remaining modules. So each run surfaces exactly one actionable failing
37	skill — handle at most one validation-failing skill per iteration.
38	- Skills whose `expected_output` YAML already exist on disk are skipped on re-runs, so each
39	iteration advances quickly to the next unresolved skill.
40
41	## Safety Rules — STOP conditions (never loop forever)
42
43	For every STOP condition, stop the automatic loop, report the evidence, offer concrete next-step
44	options, and ask the user how to proceed. Do not resume until the user answers.
45
46	- Infra / environment failure → STOP and ask. If the failure is not attributable to a specific
47	skill's logic, do not comment out a skill. Signals:
48	- `Failed: IDB lock file detected (...)`
49	- `Failed: opened binary verification failed` / `Aborting current binary after opened binary verification failure`
50	- `Failed to restore MCP connection ...`
51	- `Error: Binary file not found` or any `idalib-mcp` startup error
52	These mean the toolchain / binary / IDB is broken, not the skill. Offer relevant recovery options
53	such as clearing a confirmed stale lock, correcting the binary path, or restarting the MCP/IDA
54	session, then ask the user which action to take.
55	- `-gamever` required error → STOP and ask. The command relies on `$CS2VIBE_GAMEVER`. If the run
56	errors, offer to use the user's explicit version, read `CS2VIBE_GAMEVER` from `.env`, or inspect
57	available `configs/*.yaml` files with the user; do not guess the game version.
58	- No progress → STOP and ask. If a run's failing skill is the same one you just commented (the
59	config edit didn't take), or the `Failed` count did not go down, report the evidence and offer
60	investigation options. Never re-comment the same block or force past it.
61	- Unrelated unittest failure → STOP and ask. Only dependency gaps reported by
62	`TestConfigSkillDependencyGraph.test_config_module_skills_have_no_expected_input_order_gaps` may
63	lead to asking whether to quarantine more skills. Do not hide failures from any other test,
64	exception, import error, or test infrastructure problem by commenting out config entries. Offer
65	repair or investigation options and ask the user how to proceed.
66	- *

People who liked this also viewed

6 picks

Type

run-validation-until-no-failure

58stars10forks

View on GitHub

How to install run-validation-until-no-failure?

hlnd2t/cs2_vibesignatures/run-validation-until-no-failure

$npx -y skills add HLND2T/CS2_VibeSignatures --skill run-validation-until-no-failure

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Run Validation Until No Failure
2
3	Loop the analysis/validation pipeline until it reports `Failed: 0`. Each iteration: run
4	`ida_analyze_bin.py -debug`; if a skill failed (the pipeline is fail-fast, so it's exactly one),
5	diagnose why, present several concrete resolution options, and ask the user which option to take.
6	Do not disable or record the skill before the user answers. If the user explicitly answers that it
7	cannot be solved now and asks to temporarily disable the skill, append the failure to
8	`docs/ida_validation_failure-<GAMEVER>.md`, comment that skill out of `configs/<GAMEVER>.yaml`, and
9	run the `unit` and `repository-contract` unittest suites before resuming validation. If the config
10	dependency test reports consumers whose `expected_input` can no longer be produced, apply the same
11	ask-before-action rule to each dependent skill. Environment / infrastructure failures are not
12	skill bugs — STOP and surface those instead of quarantining a skill.
13
14	Resolve `GAMEVER` from the user's explicit request or `CS2VIBE_GAMEVER`; use only
15	`configs/$GAMEVER.yaml` for the validation run and all quarantine edits.
16
17	## When to Use
18
19	- The user asks to "run validation until it passes / until no failures", "get validation green", or
20	"disable the failing skills and keep going".
21	- After bumping to a new game version or landing many new skills, to triage a wave of failures — each
22	gets quarantined with its reason for later fixing.
23
24	## Why the loop works (pipeline facts)
25
26	- `uv run ida_analyze_bin.py -debug` ends with a summary and exits non-zero when `Failed > 0`:
27	```
28	============================================================
29	Summary
30	============================================================
31	Successful: N
32	Failed: N
33	Skipped: N
34	```
35	- It is fully fail-fast: the first failing skill aborts the remaining skills, the remaining
36	platforms, and the remaining modules. So each run surfaces exactly one actionable failing
37	skill — handle at most one validation-failing skill per iteration.
38	- Skills whose `expected_output` YAML already exist on disk are skipped on re-runs, so each
39	iteration advances quickly to the next unresolved skill.
40
41	## Safety Rules — STOP conditions (never loop forever)
42
43	For every STOP condition, stop the automatic loop, report the evidence, offer concrete next-step
44	options, and ask the user how to proceed. Do not resume until the user answers.
45
46	- Infra / environment failure → STOP and ask. If the failure is not attributable to a specific
47	skill's logic, do not comment out a skill. Signals:
48	- `Failed: IDB lock file detected (...)`
49	- `Failed: opened binary verification failed` / `Aborting current binary after opened binary verification failure`
50	- `Failed to restore MCP connection ...`
51	- `Error: Binary file not found` or any `idalib-mcp` startup error
52	These mean the toolchain / binary / IDB is broken, not the skill. Offer relevant recovery options
53	such as clearing a confirmed stale lock, correcting the binary path, or restarting the MCP/IDA
54	session, then ask the user which action to take.
55	- `-gamever` required error → STOP and ask. The command relies on `$CS2VIBE_GAMEVER`. If the run
56	errors, offer to use the user's explicit version, read `CS2VIBE_GAMEVER` from `.env`, or inspect
57	available `configs/*.yaml` files with the user; do not guess the game version.
58	- No progress → STOP and ask. If a run's failing skill is the same one you just commented (the
59	config edit didn't take), or the `Failed` count did not go down, report the evidence and offer
60	investigation options. Never re-comment the same block or force past it.
61	- Unrelated unittest failure → STOP and ask. Only dependency gaps reported by
62	`TestConfigSkillDependencyGraph.test_config_module_skills_have_no_expected_input_order_gaps` may
63	lead to asking whether to quarantine more skills. Do not hide failures from any other test,
64	exception, import error, or test infrastructure problem by commenting out config entries. Offer
65	repair or investigation options and ask the user how to proceed.
66	- *

People who liked this also viewed

6 picks

Type