Build a regression + eval harness for AI-written code and AI features. Generates characterization tests that lock current behavior before a refactor, scaffolds a Promptfoo eval suite for chatbots/RAG/classifiers, and wires it into the ship-gate. Use when the user wants evals, regression tests for AI code, to stop AI features drifting, or to test an LLM feature.
How to install evals?
houseofmvps/ultraship/evals
$npx -y skills add houseofmvps/ultraship --skill evalsInstalls into the current project.
›Prefer a prompt? Paste this to your agent
Run `npx skills use "https://github.com/houseofmvps/ultraship" --skill "houseofmvps/ultraship/evals"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.
Use the skills in "https://github.com/houseofmvps/ultraship" that are relevant to the current task. Run `npx skills add "https://github.com/houseofmvps/ultraship"` and select the relevant skills, then follow their instructions.
Related
6 picksType
- improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.Jul 2026
- domain-modelingBuild and sharpen a project's domain model. Use when the user wants to pin down domain terminology or a ubiquitous language, record an architectural decision,…Jul 2026
- caveman-reviewUltra-compressed code review comments. Cuts noise from PR feedback while preserving the actionable signal. Each comment is one line: location, problem, fix.Jul 2026
- codebase-designShared vocabulary for designing deep modules. Use when the user wants to design or improve a module's interface, find deepening opportunities, decide where a…Jul 2026
- code-reviewReview the changes since a fixed point (commit, branch, tag, or merge-base) along two axes — Standards (does the code follow this repo's documented coding…Jul 2026
- resolving-merge-conflictsUse when you need to resolve an in-progress git merge/rebase conflict.Jul 2026
