sentry-setup-ai-monitoring

Name: sentry-setup-ai-monitoring
Author: getsentry

242stars30forks

How to install sentry-setup-ai-monitoring?

getsentry/sentry-for-ai/sentry-setup-ai-monitoring

$npx -y skills add getsentry/sentry-for-ai --skill sentry-setup-ai-monitoring

Installs into the current project.

Summary

Setup Sentry AI Agent Monitoring in any project. Use when asked to monitor LLM calls, track AI agents, track conversations, or instrument OpenAI/Anthropic/Vercel AI/LangChain/Google GenAI/Pydantic AI/Laravel AI. Detects installed AI SDKs and configures appropriate integrations.

Files · 1

View on GitHub

SKILL.md

1	> [All Skills](../../SKILL_TREE.md) > [Feature Setup](../sentry-feature-setup/SKILL.md) > AI Monitoring
2
3	# Setup Sentry AI Agent Monitoring
4
5	Configure Sentry to track LLM calls, agent executions, tool usage, and token consumption.
6
7	## Invoke This Skill When
8
9	- User asks to "monitor AI/LLM calls" or "track OpenAI/Anthropic usage"
10	- User wants "AI observability" or "agent monitoring"
11	- User asks about token usage, model latency, or AI costs
12
13	Important: The SDK versions, API names, and code samples below are examples. Always verify against [docs.sentry.io](https://docs.sentry.io) before implementing, as APIs and minimum versions may have changed.
14
15	## Prerequisites
16
17	AI monitoring requires tracing enabled (`tracesSampleRate > 0`).
18
19	If the app has multi-turn chats, set a conversation ID by default anywhere it makes sense to identify a chat session. Sentry uses `gen_ai.conversation.id` to group related AI spans into Conversations. Some integrations infer it automatically, but many setups need to set it explicitly.
20
21	## Data Capture Warning
22
23	Prompt and output recording captures user content that is likely PII. In JavaScript, genAI input/output capture is on by default (governed by `dataCollection.genAI`); in Python it is enabled via `send_default_pii=True`; in Laravel it is enabled via `SENTRY_SEND_DEFAULT_PII=true`. Before relying on this capture (or per-integration overrides — `recordInputs`/`recordOutputs` in JS, `include_prompts` in Python), confirm:
24
25	- The application's privacy policy permits capturing user prompts and model responses
26	- Captured data complies with applicable regulations (GDPR, CCPA, etc.)
27	- Sentry data retention settings are appropriate for the sensitivity of the data
28
29	Ask the user whether they want prompt/output capture enabled. Do not enable prompt/output capture without explicit confirmation. Use `tracesSampleRate: 1.0` only in development; in production, use a lower value or a `tracesSampler` function.
30
31	## Detection First
32
33	Always detect installed AI SDKs before configuring:
34
35	```bash
36	# JavaScript
37	grep -E '"(openai\|@anthropic-ai/sdk\|ai\|@langchain\|@google/genai)"' package.json
38
39	# Python
40	grep -E '(openai\|anthropic\|langchain\|huggingface)' requirements.txt pyproject.toml 2>/dev/null
41
42	# PHP / Laravel
43	grep -E '"(laravel/ai\|openai-php\|openai/\|anthropic\|llm)' composer.json 2>/dev/null
44	ls artisan 2>/dev/null && echo "Laravel detected"
45	```
46
47	## Sampling Check
48
49	After detecting AI SDKs, check the current sampling configuration:
50
51	```bash
52	# JavaScript
53	grep -E 'tracesSampleRate\|tracesSampler' sentry..config. instrument.* src/instrument.* app/instrument.* 2>/dev/null
54
55	# Python
56	grep -E 'traces_sample_rate\|traces_sampler' .py /.py 2>/dev/null
57
58	# PHP / Laravel
59	grep -E 'SENTRY_TRACES_SAMPLE_RATE\|traces_sample_rate\|traces_sampler' .env config/sentry.php 2>/dev/null
60	```
61
62	If `tracesSampleRate` / `traces_sample_rate` is below 1.0 AND no `tracesSampler` / `traces_sampler` is configured:
63
64	Ask the user:
65
66	> "Your current sample rate is {rate}. Agent runs are sampled as complete span trees — if the root span is dropped, all child gen_ai spans are lost. For full AI visibility, gen_ai-related transactions should be sampled at 100%. Would you like me to set up a `tracesSampler` that keeps AI traces at 100% while sampling other traffic at your current rate?"
67
68	If user confirms, read `${SKILL_ROOT}/references/sampling.md` for implementation patterns.
69
70	## Supported SDKs
71
72	### JavaScript
73
74	\| Package \| Integration \| Min Sentry SDK \| Auto? \|
75	\|---------\|-------------\|----------------\|-------\|
76	\| `openai` \| `openAIIntegration()` \| 10.53.0 \| Yes \|
77	\| `@anthropic-ai/sdk` \| `anthropicAIIntegration()` \| 10.53.0 \| Yes \|
78	\| `ai` (Vercel) \| `vercelAIIntegration()` \| 10.53.0 \| Yes* \|
79	\| `@langchain/*` \| `langChainIntegration()` \| 10.53.0 \| Yes \|
80	\| `@langchain/langgraph` \| `langGraphIntegration()` \| 10.53.0 \| Yes \|
81	\| `@google/genai` \| `googleGenAIIntegration()` \| 10.53.0 \| Yes \|
82
83	*Vercel AI: 10.53.0+ required. Requires `experimental_telemetry` per-call.
84
85	### Python
86
87	Integrations auto-enable when the AI package is installed — no explicit registration needed:
88
89	\| Package \| Auto? \| Notes \|
90	\|---------\|-------\|-------\|
91	\| `openai` \| Yes \| Includes OpenAI Agents SDK \|
92	\| `anthropic` \| Yes \| \|
93	\| `langchain` / `langgraph` \| Yes \| \|
94	\| `huggingface_hub` \| Yes \| \|
95	\| `google-genai` \| Yes \| \|
96	\| `pydantic-ai` \| Yes \| \|
97	\| `litellm` \| No \| Requires explicit integration \|
98	\| `mcp` (Model Context Protocol) \| Yes \| \|
99
100	### PHP / Laravel
101
102	\| Package \| Integration \| Min Sentry SDK \| Auto? \|
103	\|---------\|------

People who liked this also viewed

6 picks

Type

sentry-setup-ai-monitoring

242stars30forks

View on GitHub

How to install sentry-setup-ai-monitoring?

getsentry/sentry-for-ai/sentry-setup-ai-monitoring

$npx -y skills add getsentry/sentry-for-ai --skill sentry-setup-ai-monitoring

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	> [All Skills](../../SKILL_TREE.md) > [Feature Setup](../sentry-feature-setup/SKILL.md) > AI Monitoring
2
3	# Setup Sentry AI Agent Monitoring
4
5	Configure Sentry to track LLM calls, agent executions, tool usage, and token consumption.
6
7	## Invoke This Skill When
8
9	- User asks to "monitor AI/LLM calls" or "track OpenAI/Anthropic usage"
10	- User wants "AI observability" or "agent monitoring"
11	- User asks about token usage, model latency, or AI costs
12
13	Important: The SDK versions, API names, and code samples below are examples. Always verify against [docs.sentry.io](https://docs.sentry.io) before implementing, as APIs and minimum versions may have changed.
14
15	## Prerequisites
16
17	AI monitoring requires tracing enabled (`tracesSampleRate > 0`).
18
19	If the app has multi-turn chats, set a conversation ID by default anywhere it makes sense to identify a chat session. Sentry uses `gen_ai.conversation.id` to group related AI spans into Conversations. Some integrations infer it automatically, but many setups need to set it explicitly.
20
21	## Data Capture Warning
22
23	Prompt and output recording captures user content that is likely PII. In JavaScript, genAI input/output capture is on by default (governed by `dataCollection.genAI`); in Python it is enabled via `send_default_pii=True`; in Laravel it is enabled via `SENTRY_SEND_DEFAULT_PII=true`. Before relying on this capture (or per-integration overrides — `recordInputs`/`recordOutputs` in JS, `include_prompts` in Python), confirm:
24
25	- The application's privacy policy permits capturing user prompts and model responses
26	- Captured data complies with applicable regulations (GDPR, CCPA, etc.)
27	- Sentry data retention settings are appropriate for the sensitivity of the data
28
29	Ask the user whether they want prompt/output capture enabled. Do not enable prompt/output capture without explicit confirmation. Use `tracesSampleRate: 1.0` only in development; in production, use a lower value or a `tracesSampler` function.
30
31	## Detection First
32
33	Always detect installed AI SDKs before configuring:
34
35	```bash
36	# JavaScript
37	grep -E '"(openai\|@anthropic-ai/sdk\|ai\|@langchain\|@google/genai)"' package.json
38
39	# Python
40	grep -E '(openai\|anthropic\|langchain\|huggingface)' requirements.txt pyproject.toml 2>/dev/null
41
42	# PHP / Laravel
43	grep -E '"(laravel/ai\|openai-php\|openai/\|anthropic\|llm)' composer.json 2>/dev/null
44	ls artisan 2>/dev/null && echo "Laravel detected"
45	```
46
47	## Sampling Check
48
49	After detecting AI SDKs, check the current sampling configuration:
50
51	```bash
52	# JavaScript
53	grep -E 'tracesSampleRate\|tracesSampler' sentry..config. instrument.* src/instrument.* app/instrument.* 2>/dev/null
54
55	# Python
56	grep -E 'traces_sample_rate\|traces_sampler' .py /.py 2>/dev/null
57
58	# PHP / Laravel
59	grep -E 'SENTRY_TRACES_SAMPLE_RATE\|traces_sample_rate\|traces_sampler' .env config/sentry.php 2>/dev/null
60	```
61
62	If `tracesSampleRate` / `traces_sample_rate` is below 1.0 AND no `tracesSampler` / `traces_sampler` is configured:
63
64	Ask the user:
65
66	> "Your current sample rate is {rate}. Agent runs are sampled as complete span trees — if the root span is dropped, all child gen_ai spans are lost. For full AI visibility, gen_ai-related transactions should be sampled at 100%. Would you like me to set up a `tracesSampler` that keeps AI traces at 100% while sampling other traffic at your current rate?"
67
68	If user confirms, read `${SKILL_ROOT}/references/sampling.md` for implementation patterns.
69
70	## Supported SDKs
71
72	### JavaScript
73
74	\| Package \| Integration \| Min Sentry SDK \| Auto? \|
75	\|---------\|-------------\|----------------\|-------\|
76	\| `openai` \| `openAIIntegration()` \| 10.53.0 \| Yes \|
77	\| `@anthropic-ai/sdk` \| `anthropicAIIntegration()` \| 10.53.0 \| Yes \|
78	\| `ai` (Vercel) \| `vercelAIIntegration()` \| 10.53.0 \| Yes* \|
79	\| `@langchain/*` \| `langChainIntegration()` \| 10.53.0 \| Yes \|
80	\| `@langchain/langgraph` \| `langGraphIntegration()` \| 10.53.0 \| Yes \|
81	\| `@google/genai` \| `googleGenAIIntegration()` \| 10.53.0 \| Yes \|
82
83	*Vercel AI: 10.53.0+ required. Requires `experimental_telemetry` per-call.
84
85	### Python
86
87	Integrations auto-enable when the AI package is installed — no explicit registration needed:
88
89	\| Package \| Auto? \| Notes \|
90	\|---------\|-------\|-------\|
91	\| `openai` \| Yes \| Includes OpenAI Agents SDK \|
92	\| `anthropic` \| Yes \| \|
93	\| `langchain` / `langgraph` \| Yes \| \|
94	\| `huggingface_hub` \| Yes \| \|
95	\| `google-genai` \| Yes \| \|
96	\| `pydantic-ai` \| Yes \| \|
97	\| `litellm` \| No \| Requires explicit integration \|
98	\| `mcp` (Model Context Protocol) \| Yes \| \|
99
100	### PHP / Laravel
101
102	\| Package \| Integration \| Min Sentry SDK \| Auto? \|
103	\|---------\|------

People who liked this also viewed

6 picks

Type