$npx -y skills add ClickHouse/agent-skills --skill clickstack-otel-collectorUse when a user wants to wire an OpenTelemetry collector into a Managed ClickStack service on ClickHouse Cloud, either by deploying a new local collector (Docker run or Docker Compose) or by configuring their own existing collector, then send rich synthetic telemetry and verify i
| 1 | # Set up an OpenTelemetry collector for Managed ClickStack |
| 2 | |
| 3 | This skill wires an OpenTelemetry collector into a Managed ClickStack service running on |
| 4 | ClickHouse Cloud, sends rich synthetic telemetry through it, and confirms the data is actually |
| 5 | visible in ClickStack. It uses [`clickhousectl`](https://clickhouse.com/docs/interfaces/cli) |
| 6 | for all cloud and SQL operations. |
| 7 | |
| 8 | **Scope.** This skill supports two paths, chosen in Step 0: |
| 9 | |
| 10 | 1. **Deploy a new collector** locally. You can do this **two ways: individual `docker` commands, |
| 11 | or a `docker compose` file** (recommended, fewer commands and one file to start/stop). Make the |
| 12 | user aware of both up front and let them pick in Step 0; do not assume plain `docker`. Either |
| 13 | way runs the ClickStack distribution of the collector, preconfigured for Managed ClickStack. |
| 14 | 2. **Configure your own existing collector** by adding the ClickHouse exporter configuration. |
| 15 | We give you the exact config to drop in; you reload your collector. Use this if you already |
| 16 | run a collector in a **gateway** role. |
| 17 | |
| 18 | A full Kubernetes deployment (Helm, secrets in K8s Secrets) is out of scope here; the config |
| 19 | we generate in path 2 can be applied to a collector running anywhere. |
| 20 | |
| 21 | The end state is: |
| 22 | |
| 23 | - A dedicated `hyperdx_ingest` SQL user on the target service, with exactly the grants the |
| 24 | collector needs (it creates the `otel.*` schema on first write). |
| 25 | - A collector forwarding logs, traces, and metrics into the `otel` database on the service, |
| 26 | either the new local ClickStack collector or your existing one. |
| 27 | - Rich synthetic telemetry across several services, severities, span statuses, and metric |
| 28 | types, so ClickStack's Search, Service Map, and dashboards have something real to show. |
| 29 | - The service confirmed **awake**, and the user walked through the ClickStack onboarding in the |
| 30 | Cloud console so they can actually *see* their data. |
| 31 | |
| 32 | Secrets (the OTLP auth token and the SQL password) are generated locally, written **once** to a |
| 33 | `0600` env file, and passed to Docker via `--env-file`. They are never pasted into the chat, |
| 34 | never passed with `docker run -e`, and never echoed back after creation. |
| 35 | |
| 36 | Follow these steps in order. Each step depends on state established by the previous one. |
| 37 | |
| 38 | --- |
| 39 | |
| 40 | ## Step 0: Choose your path |
| 41 | |
| 42 | Ask the user two short questions before doing anything else, because they determine which later |
| 43 | steps run. |
| 44 | |
| 45 | **Question 1: Do you already have an OpenTelemetry collector running in a gateway role?** |
| 46 | |
| 47 | - **No, set one up for me.** -> the **new-collector** path. Continue to Question 2. |
| 48 | - **Yes, I have one.** -> the **existing-collector** path. Skip Question 2 (it does not apply), |
| 49 | and in Step 6 you will configure their collector rather than deploy a new one. |
| 50 | |
| 51 | **Question 2 (new-collector path only): Run the collector with individual Docker commands, or a |
| 52 | Docker Compose file?** |
| 53 | |
| 54 | - **Docker Compose (recommended).** Fewer commands, one file to start and stop, easiest to |
| 55 | re-run. Best if `docker compose` is available. |
| 56 | - **Individual Docker commands.** Use if Compose is not installed or you prefer explicit |
| 57 | commands. |
| 58 | |
| 59 | Record the answers as `COLLECTOR_PATH` (`new` or `existing`) and, for the new path, |
| 60 | `DEPLOY_MODE` (`compose` or `run`). Refer back to them in Step 6 and Step 7. |
| 61 | |
| 62 | --- |
| 63 | |
| 64 | ## Step 1: Batch the permissions up front |
| 65 | |
| 66 | Coding agents prompt for approval the first time they see each shell command. To avoid |
| 67 | interrupting the user every few steps, ask them once, up front, to allowlist the command |
| 68 | prefixes below (the "always allow for this project / session" option in their agent). There are |
| 69 | no destructive operations and nothing targets anything outside this project or their ClickHouse |
| 70 | Cloud service. |
| 71 | |
| 72 | | Command prefix | Used for | Needed when | |
| 73 | | --- | --- | --- | |
| 74 | | `openssl rand …` | generate the OTLP token and SQL password | always | |
| 75 | | `clickhousectl cloud …` | auth, resolve the service, run SQL via the Query API | always | |
| 76 | | `jq …` | parse JSON from `clickhousectl` | always | |
| 77 | | `docker …` / `docker compose …` | run/inspect the collector and the telemetry generator | new-collector path, and the optional telemetry check | |
| 78 | | `curl …` | local health check against `localhost:13133` (and installing `clickhousectl` if missing) | new-collector path | |
| 79 | |
| 80 | Tell the user, in your own words: *"If your agent supports it, choose 'always allow' for each |
| 81 | of these the first time it asks. The whole run is read-only against your machine except for the |
| 82 | collector container, and write operations against ClickHouse are limited to creating the ingest |
| 83 | user and the `otel` schema."* |
| 84 | |
| 85 | If |