livekit-simulations

Name: livekit-simulations
Author: livekit

61stars11forks

How to install livekit-simulations?

livekit/agent-skills/livekit-simulations

$npx -y skills add livekit/agent-skills --skill livekit-simulations

Installs into the current project.

Summary

Generate targeted test scenarios for a LiveKit voice or chat agent and run them as simulations — locally, from the agent''s own code plus what the user wants stress-tested. Use whenever the user wants to "test my agent", "what should I test", "create/generate simulation scenarios

Files · 1

View on GitHub

SKILL.md

1	<!-- ============================================================
2	BETA NOTICE — TEMPORARY. Delete this whole block (down to the
3	END BETA NOTICE marker) at GA. Everything from the
4	"# Generating Simulation Scenarios" heading onward is the
5	permanent, production-oriented skill.
6	============================================================ -->
7
8	> ⚠️ Simulations are in private beta (not yet generally available).
9	> - No docs/MCP coverage yet. For the `lk agent simulate` command surface,
10	> use `lk agent simulate --help` and the LiveKit Cloud dashboard rather than
11	> `lk docs` / MCP until simulations are documented.
12	> - Recent SDK required. Running simulations needs the 1.6 line of
13	> `livekit-agents`. Confirm the installed version rather than assuming.
14	> - Limited availability / auth. Creating runs needs the project enabled for
15	> simulations and a current `lk cloud auth` session. (Generating scenarios —
16	> the main job of this skill — needs neither; it's fully local.)
17
18	<!-- ===================== END BETA NOTICE ====================== -->
19
20	# Generating Simulation Scenarios
21
22	The most valuable thing you can do with simulations is generate good test scenarios for the user's agent — grounded in the agent's actual code and in what the user wants stress-tested — then run them. You do this locally: you read the code with your normal tools (nothing is uploaded), and you (the coding agent) are the model that does the generation, so no extra API keys or services are needed.
23
24	A scenario = a simulated user's persona + goals (`instructions`) and the pass criteria (`agent_expectations`). A simulation plays each scenario against the agent over text and an LLM judge scores it. Your job is to produce a high-quality, diverse, on-target set of scenarios and write them to a YAML scenarios file the CLI can run.
25
26	## What makes this better than autopilot
27	A naive "just generate some tests" misses the point. Three things make this skill worth using:
28	1. It reads the agent's real code — so scenarios respect what the agent can actually do and where it blocks (especially constraints/unavailable items), instead of guessing from the name.
29	2. It is steered by the user. The user knows what they're worried about. Always capture that intent and thread it through. This is the headline — see `references/user-guidance.md`.
30	3. It guarantees coverage of every risk. Left alone, generation drifts to plausible happy-path calls and silently skips the hard cases — withholding a required field, supplying an invalid value, an empty lookup, and the guardrail/abuse surface (out-of-scope, harmful, professional-advice, sensitive-data, prompt-extraction). This skill turns the agent's constraints into an explicit risk checklist and requires at least one scenario per item — see `references/analyzing-the-agent.md` and `references/writing-scenarios.md`.
31
32	## The flow
33
34	1. Describe the agent + build the risk checklist — read its code locally and write a test-oriented description (Identity / Capabilities / Constraints) to `description.md`, and an explicit risk checklist to `risks.yaml` (one entry per must-test constraint/guardrail, each with an `id` and `category`). Follow `references/analyzing-the-agent.md`. Never upload the code.
35	2. Get the user's test focus — if they didn't say what to probe, ask. Apply it per `references/user-guidance.md` (append a `# Test Focus` to `description.md`, and bias authoring). Focus is additive — it deepens chosen risks but never drops the per-risk coverage floor. If they truly have no preference, generate broad and say so.
36	3. Author the scenarios — at least one per risk — write a diverse set of ~10 scenarios grounded in `description.md` and the focus, generating the persona / mood / situation variety from your own judgment (this version ships no attribute libraries). Guarantee coverage: every `risks.yaml` item gets ≥1 dedicated scenario, written with the shape that actually exercises

People who liked this also viewed

6 picks

Type

livekit-simulations

61stars11forks

View on GitHub

How to install livekit-simulations?

livekit/agent-skills/livekit-simulations

$npx -y skills add livekit/agent-skills --skill livekit-simulations

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	<!-- ============================================================
2	BETA NOTICE — TEMPORARY. Delete this whole block (down to the
3	END BETA NOTICE marker) at GA. Everything from the
4	"# Generating Simulation Scenarios" heading onward is the
5	permanent, production-oriented skill.
6	============================================================ -->
7
8	> ⚠️ Simulations are in private beta (not yet generally available).
9	> - No docs/MCP coverage yet. For the `lk agent simulate` command surface,
10	> use `lk agent simulate --help` and the LiveKit Cloud dashboard rather than
11	> `lk docs` / MCP until simulations are documented.
12	> - Recent SDK required. Running simulations needs the 1.6 line of
13	> `livekit-agents`. Confirm the installed version rather than assuming.
14	> - Limited availability / auth. Creating runs needs the project enabled for
15	> simulations and a current `lk cloud auth` session. (Generating scenarios —
16	> the main job of this skill — needs neither; it's fully local.)
17
18	<!-- ===================== END BETA NOTICE ====================== -->
19
20	# Generating Simulation Scenarios
21
22	The most valuable thing you can do with simulations is generate good test scenarios for the user's agent — grounded in the agent's actual code and in what the user wants stress-tested — then run them. You do this locally: you read the code with your normal tools (nothing is uploaded), and you (the coding agent) are the model that does the generation, so no extra API keys or services are needed.
23
24	A scenario = a simulated user's persona + goals (`instructions`) and the pass criteria (`agent_expectations`). A simulation plays each scenario against the agent over text and an LLM judge scores it. Your job is to produce a high-quality, diverse, on-target set of scenarios and write them to a YAML scenarios file the CLI can run.
25
26	## What makes this better than autopilot
27	A naive "just generate some tests" misses the point. Three things make this skill worth using:
28	1. It reads the agent's real code — so scenarios respect what the agent can actually do and where it blocks (especially constraints/unavailable items), instead of guessing from the name.
29	2. It is steered by the user. The user knows what they're worried about. Always capture that intent and thread it through. This is the headline — see `references/user-guidance.md`.
30	3. It guarantees coverage of every risk. Left alone, generation drifts to plausible happy-path calls and silently skips the hard cases — withholding a required field, supplying an invalid value, an empty lookup, and the guardrail/abuse surface (out-of-scope, harmful, professional-advice, sensitive-data, prompt-extraction). This skill turns the agent's constraints into an explicit risk checklist and requires at least one scenario per item — see `references/analyzing-the-agent.md` and `references/writing-scenarios.md`.
31
32	## The flow
33
34	1. Describe the agent + build the risk checklist — read its code locally and write a test-oriented description (Identity / Capabilities / Constraints) to `description.md`, and an explicit risk checklist to `risks.yaml` (one entry per must-test constraint/guardrail, each with an `id` and `category`). Follow `references/analyzing-the-agent.md`. Never upload the code.
35	2. Get the user's test focus — if they didn't say what to probe, ask. Apply it per `references/user-guidance.md` (append a `# Test Focus` to `description.md`, and bias authoring). Focus is additive — it deepens chosen risks but never drops the per-risk coverage floor. If they truly have no preference, generate broad and say so.
36	3. Author the scenarios — at least one per risk — write a diverse set of ~10 scenarios grounded in `description.md` and the focus, generating the persona / mood / situation variety from your own judgment (this version ships no attribute libraries). Guarantee coverage: every `risks.yaml` item gets ≥1 dedicated scenario, written with the shape that actually exercises

People who liked this also viewed

6 picks

Type