observability

Name: observability
Author: DevelopersGlobal

64stars10forks

How to install observability?

developersglobal/ai-agent-skills/observability

$npx -y skills add DevelopersGlobal/ai-agent-skills --skill observability

Installs into the current project.

Summary

Structured logging, distributed tracing, and alerting for AI systems and traditional services. You can't fix what you can't see.

Files · 1

View on GitHub

SKILL.md

1	## Overview
2
3	Observability is the ability to understand the internal state of a system from its external outputs. For AI systems this is especially critical: agents make decisions that are hard to interpret without detailed telemetry.
4
5	The three pillars: Logs (what happened), Traces (how long and where), Metrics (aggregate health).
6
7	## When to Use
8
9	- Before deploying any new service to production
10	- When adding AI agent capabilities to an existing system
11	- When debugging production issues
12	- When designing multi-agent pipelines
13
14	## Process
15
16	### Step 1: Structured Logging
17
18	1. All logs must be structured (JSON, not free text). Fields: `timestamp`, `level`, `service`, `traceId`, `message`, `context`.
19	2. Log levels used correctly:
20	- `ERROR`: Something failed that requires immediate attention
21	- `WARN`: Something unexpected happened but the system recovered
22	- `INFO`: Normal significant events (requests received, jobs completed)
23	- `DEBUG`: Detailed diagnostic information (off in production by default)
24	3. Never log secrets, PII, or auth tokens.
25	4. For AI systems, log: prompt inputs (sanitized), model outputs, token counts, latency, model version.
26
27	Verify: Logs are structured JSON. No secrets in logs. AI interactions logged.
28
29	### Step 2: Distributed Tracing
30
31	5. Every request gets a unique `traceId` generated at the entry point.
32	6. `traceId` is propagated through all downstream calls (HTTP headers, message queues, agent calls).
33	7. Each service/agent creates a span for its work, with: start time, end time, parent span ID.
34	8. Use OpenTelemetry as the standard instrumentation library.
35
36	Verify: You can trace a single request across all services/agents in a single view.
37
38	### Step 3: Metrics
39
40	9. Define and track key metrics:
41	- RED metrics: Rate (requests/sec), Errors (error rate %), Duration (latency p50/p95/p99)
42	- AI-specific: Token usage, prompt cost, model latency, hallucination rate, retrieval precision
43	10. Dashboards: one dashboard per service with RED metrics, one dashboard for AI system health.
44
45	Verify: RED metrics are tracked for every service. AI-specific metrics tracked for AI systems.
46
47	### Step 4: Alerting
48
49	11. Alerts must be actionable — every alert should have a runbook.
50	12. Alert on symptoms (high error rate, high latency), not just causes.
51	13. AI-specific alerts: token budget exceeded, model error rate spike, retrieval failure rate spike.
52	14. On-call rotation: someone is responsible for every alert at all times.
53
54	Verify: Every alert has a runbook. On-call rotation defined.
55
56	## Common Rationalizations (and Rebuttals)
57
58	\| Excuse \| Rebuttal \|
59	\|--------\|----------\|
60	\| "We'll add monitoring after launch" \| You'll be fighting fires blind. Add it before. \|
61	\| "Console.log is enough" \| In production, console.log is noise. Structured logs with context are signals. \|
62	\| "The AI model handles it internally" \| Model internals are a black box. You must observe the inputs and outputs. \|
63
64	## Verification
65
66	- [ ] Structured JSON logging on all services
67	- [ ] No secrets in logs
68	- [ ] Distributed tracing with trace ID propagation
69	- [ ] RED metrics tracked for all services
70	- [ ] AI-specific metrics tracked (tokens, cost, latency)
71	- [ ] Alerts configured with runbooks
72
73	## References
74
75	- [production-deployment skill](../production-deployment/SKILL.md)
76	- [multi-agent-orchestration skill](../multi-agent-orchestration/SKILL.md)
77	- OpenTelemetry documentation

People who liked this also viewed

6 picks

Type

find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 20262.6M27k
grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026633k182k
vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026573k29k
improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026522k182k
setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026454k182k
triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026424k182k

observability

64stars10forks

View on GitHub

How to install observability?

developersglobal/ai-agent-skills/observability

$npx -y skills add DevelopersGlobal/ai-agent-skills --skill observability

Installs into the current project.

Summary

Structured logging, distributed tracing, and alerting for AI systems and traditional services. You can't fix what you can't see.

Files · 1

View on GitHub

SKILL.md

1	## Overview
2
3	Observability is the ability to understand the internal state of a system from its external outputs. For AI systems this is especially critical: agents make decisions that are hard to interpret without detailed telemetry.
4
5	The three pillars: Logs (what happened), Traces (how long and where), Metrics (aggregate health).
6
7	## When to Use
8
9	- Before deploying any new service to production
10	- When adding AI agent capabilities to an existing system
11	- When debugging production issues
12	- When designing multi-agent pipelines
13
14	## Process
15
16	### Step 1: Structured Logging
17
18	1. All logs must be structured (JSON, not free text). Fields: `timestamp`, `level`, `service`, `traceId`, `message`, `context`.
19	2. Log levels used correctly:
20	- `ERROR`: Something failed that requires immediate attention
21	- `WARN`: Something unexpected happened but the system recovered
22	- `INFO`: Normal significant events (requests received, jobs completed)
23	- `DEBUG`: Detailed diagnostic information (off in production by default)
24	3. Never log secrets, PII, or auth tokens.
25	4. For AI systems, log: prompt inputs (sanitized), model outputs, token counts, latency, model version.
26
27	Verify: Logs are structured JSON. No secrets in logs. AI interactions logged.
28
29	### Step 2: Distributed Tracing
30
31	5. Every request gets a unique `traceId` generated at the entry point.
32	6. `traceId` is propagated through all downstream calls (HTTP headers, message queues, agent calls).
33	7. Each service/agent creates a span for its work, with: start time, end time, parent span ID.
34	8. Use OpenTelemetry as the standard instrumentation library.
35
36	Verify: You can trace a single request across all services/agents in a single view.
37
38	### Step 3: Metrics
39
40	9. Define and track key metrics:
41	- RED metrics: Rate (requests/sec), Errors (error rate %), Duration (latency p50/p95/p99)
42	- AI-specific: Token usage, prompt cost, model latency, hallucination rate, retrieval precision
43	10. Dashboards: one dashboard per service with RED metrics, one dashboard for AI system health.
44
45	Verify: RED metrics are tracked for every service. AI-specific metrics tracked for AI systems.
46
47	### Step 4: Alerting
48
49	11. Alerts must be actionable — every alert should have a runbook.
50	12. Alert on symptoms (high error rate, high latency), not just causes.
51	13. AI-specific alerts: token budget exceeded, model error rate spike, retrieval failure rate spike.
52	14. On-call rotation: someone is responsible for every alert at all times.
53
54	Verify: Every alert has a runbook. On-call rotation defined.
55
56	## Common Rationalizations (and Rebuttals)
57
58	\| Excuse \| Rebuttal \|
59	\|--------\|----------\|
60	\| "We'll add monitoring after launch" \| You'll be fighting fires blind. Add it before. \|
61	\| "Console.log is enough" \| In production, console.log is noise. Structured logs with context are signals. \|
62	\| "The AI model handles it internally" \| Model internals are a black box. You must observe the inputs and outputs. \|
63
64	## Verification
65
66	- [ ] Structured JSON logging on all services
67	- [ ] No secrets in logs
68	- [ ] Distributed tracing with trace ID propagation
69	- [ ] RED metrics tracked for all services
70	- [ ] AI-specific metrics tracked (tokens, cost, latency)
71	- [ ] Alerts configured with runbooks
72
73	## References
74
75	- [production-deployment skill](../production-deployment/SKILL.md)
76	- [multi-agent-orchestration skill](../multi-agent-orchestration/SKILL.md)
77	- OpenTelemetry documentation

People who liked this also viewed

6 picks

Type