configs-variations
starsforks
How to install configs-variations?
launchdarkly/ai-tooling/configs-variations
$npx -y skills add launchdarkly/ai-tooling --skill configs-variationsExperiment with configs by creating and managing variations. Helps you test different models, prompts, and parameters to find what works best through systematic experimentation.
SKILL.md
| 1 | # Config Variations |
| 2 | |
| 3 | You're using a skill that will guide you through testing and optimizing configs through variations. Your job is to design experiments, create variations, and systematically find what works best. |
| 4 | |
| 5 | ## Prerequisites |
| 6 | |
| 7 | This skill requires the remotely hosted LaunchDarkly MCP server to be configured in your environment. |
| 8 | |
| 9 | **Primary MCP tool:** |
| 10 | - `clone-ai-config-variation` -- clone a baseline variation with selective overrides (recommended for experimentation) |
| 11 | |
| 12 | **Alternative MCP tools (for more control):** |
| 13 | - `get-ai-config` -- review existing variations before adding new ones |
| 14 | - `create-ai-config-variation` -- create new variations from scratch |
| 15 | |
| 16 | **Optional MCP tools:** |
| 17 | - `update-ai-config-variation` -- refine a variation after creation |
| 18 | - `delete-ai-config-variation` -- remove variations that didn't work out |
| 19 | |
| 20 | ## Core Principles |
| 21 | |
| 22 | 1. **Test One Thing at a Time**: Change model OR prompt OR parameters, not all at once |
| 23 | 2. **Have a Hypothesis**: Know what you're trying to improve |
| 24 | 3. **Measure Results**: Use metrics to compare variations |
| 25 | 4. **Verify via Tool**: The agent fetches the config to confirm variations exist |
| 26 | |
| 27 | ## Workflow |
| 28 | |
| 29 | ### Step 1: Identify What to Optimize |
| 30 | |
| 31 | What's the problem? Cost, quality, speed, accuracy? How will you measure success? |
| 32 | |
| 33 | ### Step 2: Design the Experiment |
| 34 | |
| 35 | | Goal | What to Vary | |
| 36 | |------|--------------| |
| 37 | | Reduce cost | Cheaper model (e.g., `gpt-4o-mini`) | |
| 38 | | Improve quality | Better model or more detailed prompt | |
| 39 | | Reduce latency | Faster model, lower `max_tokens` | |
| 40 | | Increase accuracy | Different model family (Claude vs GPT-4) | |
| 41 | |
| 42 | ### Step 3: Create Variations (Recommended: Clone with Overrides) |
| 43 | |
| 44 | Use `clone-ai-config-variation` to duplicate the baseline and override only what you're testing. The tool reads the source variation, merges your overrides, and creates the new variation. Everything you **don't** pass is inherited from the source automatically. |
| 45 | |
| 46 | **Required fields:** |
| 47 | - `sourceVariationKey` -- the baseline to clone from |
| 48 | - `key` and `name` -- identifiers for the new variation (e.g., `gpt4o-mini-cost-test`) |
| 49 | |
| 50 | **Override ONLY the fields you are testing.** Leave all other fields unset -- do not pass them even if you know their current values. The clone tool inherits them from the source. This enforces the one-variable-at-a-time principle: |
| 51 | |
| 52 | - Testing a cheaper model? Pass only `modelConfigKey` and `modelName`. Do NOT pass `instructions`, `messages`, or `parameters`. |
| 53 | - Testing different instructions? Pass only `instructions`. Do NOT pass `modelConfigKey` or `modelName`. |
| 54 | - Testing a parameter? Pass only `parameters`. Do NOT pass model or prompt fields. |
| 55 | |
| 56 | The response returns both the source and created variation, so you can immediately verify the diff. |
| 57 | |
| 58 | ### Step 3 (Alternative): Create from Scratch |
| 59 | |
| 60 | If you need full control, use `get-ai-config` first to review the current state, then `create-ai-config-variation` with all fields specified manually. Always fetch before creating so you understand the existing config's mode, model, and parameters. |
| 61 | |
| 62 | ### Step 4: Verify |
| 63 | |
| 64 | If you used `clone-ai-config-variation`, the response includes both source and created variations for immediate comparison. Otherwise, use `get-ai-config` to confirm. |
| 65 | |
| 66 | **Report results:** |
| 67 | - Variations created with correct models and parameters |
| 68 | - Only the intended variable differs between variations |
| 69 | - Flag any issues |
| 70 | |
| 71 | **Note on API responses:** After calling a creation or clone tool, treat a successful response as confirmation that the operation succeeded. The API response may not echo back every field you sent (e.g., model fields may show defaults). Do not retry or assume failure based on response field values alone -- verify with `get-ai-config` if needed. |
| 72 | |
| 73 | ## modelConfigKey Format |
| 74 | |
| 75 | Required for models to display in the UI. Format: `{Provider}.{model-id}`: |
| 76 | - `OpenAI.gpt-4o`, `OpenAI.gpt-4o-mini` |
| 77 | - `Anthropic.claude-sonnet-4-5`, `Anthropic.claude-3-5-sonnet` |
| 78 | |
| 79 | ## Safety: Protect the Baseline |
| 80 | |
| 81 | When the user wants to try a different model, prompt, or parameters, **always create a new variation alongside the baseline**. Never modify or delete the existing baseline variation. This applies even if the user says "replace" or "switch" -- the correct action is to create a new variation and let targeting/rollouts control traffic, not to edit the original. |
| 82 | |
| 83 | - Use `clone-ai-config-variation` or `create-ai-config-variation` to add the new variation |
| 84 | - Do NOT use `update-ai-config-variation` on the baseline to change its model or instructions |
| 85 | - Do NOT use `delete-ai-config-variation` on the baseline |
| 86 | - Explain to the user that keeping the baseline enables co |
People who liked this also viewed
6 picksType
- find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 2026
- grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026
- vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026
- improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026
- setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026
- triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026
