agents-optimize

Name: agents-optimize
Author: aws

586installs190forks

How to install agents-optimize?

aws/agent-toolkit-for-aws/agents-optimize

$npx -y skills add aws/agent-toolkit-for-aws --skill agents-optimize

Installs into the current project.

Summary

Use when measuring or improving agent quality and performance — set up evaluators, online monitoring, CI/CD quality gates, observability, or cost optimization. Triggers on: "evaluate my agent", "add evaluator", "measure quality", "quality gate", "run evals", "agent too slow", "wh

Files · 1

View on GitHub

SKILL.md

1	# optimize
2
3	Measure and improve your AgentCore agent's quality through evaluation, monitoring, and observability.
4
5	## When to use
6
7	- You want to know if your agent is giving good answers
8	- You want to set up continuous quality monitoring in production
9	- You want to add a quality gate to your CI/CD pipeline
10	- You want to understand agent behavior through logs, metrics, and traces
11	- You want to set up CloudWatch dashboards or X-Ray tracing
12
13	Do NOT use for:
14
15	- Debugging a specific broken agent (wrong answers, errors) → use `agents-debug`
16	- Production security hardening (IAM, auth) → use `agents-harden`
17
18	## Input
19
20	`$ARGUMENTS` can be:
21
22	- An eval goal: "add a quality gate", "set up monitoring"
23	- An observability goal: "set up CloudWatch dashboard", "understand my traces"
24	- A specific evaluator: "llm-as-a-judge", "code-based"
25	- Empty — the skill will guide based on project context
26
27	## Process
28
29	### Step 0: Verify CLI version
30
31	Run `agentcore --version`. This skill requires v0.9.0 or later.
32
33	### Step 1: Read project context
34
35	Read `agentcore/agentcore.json` to understand existing evaluators, online eval configs, and agent setup.
36
37	If `agentcore/agentcore.json` is not found:
38	> "This skill requires an AgentCore project. Use `agents-get-started` to create one."
39
40	### Step 2: Determine the workflow
41
42	\| Developer intent \| Action \|
43	\|---\|---\|
44	\| Measure quality, add evaluator, run eval, CI/CD gate, online monitoring \| Load [`references/evals.md`](references/evals.md) and follow its workflow \|
45	\| Set up observability, CloudWatch, X-Ray, logs, metrics, dashboards \| Load [`references/observability.md`](references/observability.md) and follow its workflow \|
46	\| Understand or reduce AgentCore costs \| Load [`references/cost.md`](references/cost.md) \|
47	\| Both — "I want to understand and improve my agent" \| Start with observability setup, then add evals \|
48
49	### Step 3: Follow the loaded reference
50
51	The reference file contains the full procedure. Follow it step by step.
52
53	### Cross-references
54
55	- After setting up evals, suggest `agents-harden` for production readiness
56	- If eval results reveal agent issues, suggest `agents-debug` for root cause analysis
57	- If the developer needs to add capabilities first, suggest `agents-build`
58
59	## Output
60
61	Depends on the workflow — see the loaded reference for specific outputs.
62
63	## Quality criteria
64
65	- Evaluator configuration uses only valid CLI flags
66	- Online eval sampling rate is appropriate (not 100% in production without discussion)
67	- CI/CD quality gate has a clear pass/fail threshold
68	- Observability setup includes both tracing and logging
69	- The developer understands the eval data delay: ~10 seconds put-to-get, end-to-end — one ingestion step covers both trace reads and eval queries; there is no separate indexing wait

People who liked this also viewed

6 picks

Type

agents-optimize

586installs190forks

View on GitHub

How to install agents-optimize?

aws/agent-toolkit-for-aws/agents-optimize

$npx -y skills add aws/agent-toolkit-for-aws --skill agents-optimize

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# optimize
2
3	Measure and improve your AgentCore agent's quality through evaluation, monitoring, and observability.
4
5	## When to use
6
7	- You want to know if your agent is giving good answers
8	- You want to set up continuous quality monitoring in production
9	- You want to add a quality gate to your CI/CD pipeline
10	- You want to understand agent behavior through logs, metrics, and traces
11	- You want to set up CloudWatch dashboards or X-Ray tracing
12
13	Do NOT use for:
14
15	- Debugging a specific broken agent (wrong answers, errors) → use `agents-debug`
16	- Production security hardening (IAM, auth) → use `agents-harden`
17
18	## Input
19
20	`$ARGUMENTS` can be:
21
22	- An eval goal: "add a quality gate", "set up monitoring"
23	- An observability goal: "set up CloudWatch dashboard", "understand my traces"
24	- A specific evaluator: "llm-as-a-judge", "code-based"
25	- Empty — the skill will guide based on project context
26
27	## Process
28
29	### Step 0: Verify CLI version
30
31	Run `agentcore --version`. This skill requires v0.9.0 or later.
32
33	### Step 1: Read project context
34
35	Read `agentcore/agentcore.json` to understand existing evaluators, online eval configs, and agent setup.
36
37	If `agentcore/agentcore.json` is not found:
38	> "This skill requires an AgentCore project. Use `agents-get-started` to create one."
39
40	### Step 2: Determine the workflow
41
42	\| Developer intent \| Action \|
43	\|---\|---\|
44	\| Measure quality, add evaluator, run eval, CI/CD gate, online monitoring \| Load [`references/evals.md`](references/evals.md) and follow its workflow \|
45	\| Set up observability, CloudWatch, X-Ray, logs, metrics, dashboards \| Load [`references/observability.md`](references/observability.md) and follow its workflow \|
46	\| Understand or reduce AgentCore costs \| Load [`references/cost.md`](references/cost.md) \|
47	\| Both — "I want to understand and improve my agent" \| Start with observability setup, then add evals \|
48
49	### Step 3: Follow the loaded reference
50
51	The reference file contains the full procedure. Follow it step by step.
52
53	### Cross-references
54
55	- After setting up evals, suggest `agents-harden` for production readiness
56	- If eval results reveal agent issues, suggest `agents-debug` for root cause analysis
57	- If the developer needs to add capabilities first, suggest `agents-build`
58
59	## Output
60
61	Depends on the workflow — see the loaded reference for specific outputs.
62
63	## Quality criteria
64
65	- Evaluator configuration uses only valid CLI flags
66	- Online eval sampling rate is appropriate (not 100% in production without discussion)
67	- CI/CD quality gate has a clear pass/fail threshold
68	- Observability setup includes both tracing and logging
69	- The developer understands the eval data delay: ~10 seconds put-to-get, end-to-end — one ingestion step covers both trace reads and eval queries; there is no separate indexing wait

People who liked this also viewed

6 picks

Type