Use this to test an LLM change (new prompt, new model, new retrieval) on real traffic before rolling it out to everyone. Trigger on "A/B test my prompt", "roll out a new model safely", "compare two prompts in production", "canary this change", "does this actually improve things for real users". Measure impact on real users, gated, before a full switch.
$npx -y skills add contextjet-ai/awesome-llm-observability --skill set-up-ab-testingInstalls into the current project.
Run `npx skills use "https://github.com/contextjet-ai/awesome-llm-observability" --skill "contextjet-ai/awesome-llm-observability/set-up-ab-testing"` and follow the generated skill instructions now. Read its complete output, redirecting it to a temporary file first if necessary. Resolve relative paths from the supporting-files directory it provides.
Use the skills in "https://github.com/contextjet-ai/awesome-llm-observability" that are relevant to the current task. Run `npx skills add "https://github.com/contextjet-ai/awesome-llm-observability"` and select the relevant skills, then follow their instructions.