$npx -y skills add aws/agent-toolkit-for-aws --skill aws-observabilityBuilds, configures, debugs, and optimizes AWS observability with CloudWatch (Log Insights, Metrics, Alarms, Dashboards, EMF), X-Ray, CloudTrail, and ADOT (AWS Distro for OpenTelemetry), AND enables/onboards services to Application Signals using ADOT auto-instrumentation SDKs. Cov
| 1 | # AWS Observability |
| 2 | |
| 3 | ## Overview |
| 4 | |
| 5 | Domain expertise for AWS observability across metrics, logs, and traces, covering the full lifecycle: **enabling/onboarding** a service to Application Signals using ADOT (AWS Distro for OpenTelemetry) auto-instrumentation SDKs and ServiceEvents — making the service show up in Application Signals — on EC2, ECS, EKS, and Lambda in Python, Node.js, Java, and .NET. |
| 6 | |
| 7 | **Works best with** the [AWS MCP server](https://docs.aws.amazon.com/aws-mcp/) — enables running CLI commands, querying CloudWatch, and validating configurations directly. All guidance also works with standard AWS CLI access. |
| 8 | |
| 9 | **Note:** Reference files contain specific runtime versions, quota values, and feature matrices that may change. When precision matters (e.g., deploying to production, choosing a runtime, or checking a quota), confirm values against current AWS documentation rather than relying solely on the values in these files. |
| 10 | |
| 11 | ## Routing |
| 12 | |
| 13 | | User need | Action | |
| 14 | |-----------|--------| |
| 15 | | Enabling/onboarding a service to Application Signals (auto-instrumentation) | Read [application-signals-onboarding.md](references/application-signals-onboarding.md) | |
| 16 | | Propagating ServiceEvents git/deployment metadata through CI/CD | Read [application-signals-cicd-metadata.md](references/application-signals-cicd-metadata.md) | |
| 17 | | Per-platform/per-language enablement steps | Read the matching `references/appsignals-guides/<platform>-<language>.md` (e.g. [eks-python.md](references/appsignals-guides/eks-python.md)) | |
| 18 | | Writing Log Insights queries | Read [log-insights.md](references/log-insights.md) | |
| 19 | | Configuring alarms (metric, composite, anomaly) | Read [alarms.md](references/alarms.md) | |
| 20 | | Publishing custom metrics or using EMF | Read [metrics.md](references/metrics.md) | |
| 21 | | Setting up X-Ray tracing or ADOT | Read [tracing.md](references/tracing.md) | |
| 22 | | Building dashboards | Read [dashboards.md](references/dashboards.md) | |
| 23 | | Debugging observability issues | Read [troubleshooting.md](references/troubleshooting.md) — starts with the 5 most common fixes | |
| 24 | | Debugging canary failures | Read [synthetics.md](references/synthetics.md) — see Common failures table | |
| 25 | | CloudTrail operational auditing | Read [cloudtrail.md](references/cloudtrail.md) | |
| 26 | | Setting up Lambda monitoring with CDK | Use [alarm-template.ts](assets/alarm-template.ts) as a starting point | |
| 27 | | Creating synthetic canaries | Read [synthetics.md](references/synthetics.md) | |
| 28 | | Configuring ADOT collector | Use [otel-config.yaml](assets/otel-config.yaml) as a starting point | |
| 29 | | Debugging a running service with breakpoints/snapshots — Dynamic Instrumentation (**modifies live services and capture live data**) | Read [dynamic-instrumentation.md](references/dynamic-instrumentation.md) in full before acting. Confirm with the user before any create/delete, and narrate before significant actions: observation → hypothesis → proposed action → expected result. Diagnosing running-service root cause from source/code inspection. Source inspection alone identifies hypotheses, not confirmed root causes. Keep suspected causes tentative until runtime evidence confirms them. | |
| 30 | | Spans multiple areas | Read the most specific reference first, then consult others as needed | |
| 31 | |
| 32 | ## Files |
| 33 | |
| 34 | | File | Content | |
| 35 | |------|---------| |
| 36 | | [application-signals-onboarding.md](references/application-signals-onboarding.md) | Enable Application Signals auto-instrumentation: EKS add-on, CloudWatch Agent IAM, OTLP endpoints, ServiceEvents env vars, Dynamic Instrumentation — two-tier scope by platform/language | |
| 37 | | [application-signals-cicd-metadata.md](references/application-signals-cicd-metadata.md) | ServiceEvents git & deployment metadata propagation through CI/CD (the 5 `OTEL_AWS_SERVICE_EVENTS_*` vars) | |
| 38 | | `references/appsignals-guides/` (e.g. [eks-python.md](references/appsignals-guides/eks-python.md)) | 16 per-platform × per-language enablement guides (EC2/ECS/EKS/ |