$npx -y skills add michellemayes/meta-doctor --skill add-doctor-runbookAdds a new runbook to an existing doctor and registers it in the runbook index and mode config. Use when extending a doctor's audit coverage with a new check, without re-scaffolding the entire doctor.
| 1 | # Add Doctor Runbook |
| 2 | |
| 3 | Extends an existing doctor (scaffolded by `scaffold-doctor`) with one new |
| 4 | runbook — the internal markdown audit definition that drives a single health |
| 5 | check in the doctor's agentic loop. |
| 6 | |
| 7 | --- |
| 8 | |
| 9 | ## What you need before starting |
| 10 | |
| 11 | - The doctor's project root (where `scaffold-doctor` deposited its output). |
| 12 | - A clear sense of what the new runbook should audit, what signals it reads, |
| 13 | and what a violation looks like versus expected noise. |
| 14 | |
| 15 | --- |
| 16 | |
| 17 | ## Flow |
| 18 | |
| 19 | ### 1. Locate the doctor's runbook directory and registration files |
| 20 | |
| 21 | Inside the scaffolded doctor, runbooks live in `skills/`. The two registration |
| 22 | files are: |
| 23 | |
| 24 | | File | Purpose | |
| 25 | |---|---| |
| 26 | | `skills/index.ts` | Runbook index — imports every runbook and exports a `bundledSkills` record. | |
| 27 | | `config.ts` | Mode config — assigns each runbook to `runbooksDaily` or `runbooksWeekly`. | |
| 28 | |
| 29 | Read both files before writing anything. Confirm the existing runbook names and |
| 30 | the mode lists so you can slot the new runbook in correctly. |
| 31 | |
| 32 | ### 2. Gather runbook requirements |
| 33 | |
| 34 | Ask the user (or derive from context) three things: |
| 35 | |
| 36 | 1. **Purpose** — what aspect of app health does this runbook audit? One or two |
| 37 | sentences. |
| 38 | 2. **Signals and tools** — which data sources, queries, or MCP tools will the |
| 39 | runbook call? (e.g., CloudWatch log groups, a Looker query, a DB table.) |
| 40 | 3. **Healthy vs. violation** — what does a clean run look like, and what |
| 41 | specific conditions should trigger `emit_finding`? |
| 42 | |
| 43 | Do not proceed to authoring until all three are clear. If any is ambiguous, |
| 44 | ask a targeted question — a vague boundary between healthy and violation is |
| 45 | the most common source of miscalibrated runbooks. |
| 46 | |
| 47 | ### 3. Author the runbook markdown |
| 48 | |
| 49 | Create `skills/<kebab-case-name>.md` following the structure in |
| 50 | [`../scaffold-doctor/references/writing-runbooks.md`](../scaffold-doctor/references/writing-runbooks.md). |
| 51 | |
| 52 | Key requirements (enforced by the authoring checklist in that reference): |
| 53 | |
| 54 | - **Pull live config first.** Step 1 of the Inspect section always reads |
| 55 | thresholds from the app's config store. No hardcoded numbers anywhere. |
| 56 | - **Parameterize every check.** Time windows, count thresholds, feature-flag |
| 57 | gates — all derived from the config you just read. |
| 58 | - **Healthy runs emit zero findings.** If the runbook would fire on a normal |
| 59 | day, the threshold or classify logic is wrong. |
| 60 | - **Separate violations from visibility.** Informational observations (counts, |
| 61 | rates, scorecards) go in the narrative; only actionable violations reach |
| 62 | `emit_finding`. |
| 63 | |
| 64 | For every `emit_finding` call, assign a **stable signature** using the format |
| 65 | for the finding's category, per |
| 66 | [`../scaffold-doctor/references/finding-contract.md`](../scaffold-doctor/references/finding-contract.md): |
| 67 | |
| 68 | | Category | Signature format | |
| 69 | |---|---| |
| 70 | | Code bug with stack frame | `{ExceptionClass}:{relativeFilePath}:{lineNumber}` | |
| 71 | | Code bug without stack frame | `{ExceptionClass}:unknown:{md5_12chars_of_normalized_message}` | |
| 72 | | Audit finding | `{runbook}:{category}:{bucketDate}` (ISO week / date / month) | |
| 73 | |
| 74 | Unstable signatures cause duplicate PRs and tickets. When in doubt, err toward |
| 75 | broader bucketing — false dedups are cheaper than duplicate work items. |
| 76 | |
| 77 | ### 4. Register the runbook |
| 78 | |
| 79 | Two edits required — both must be made before the doctor is redeployed. |
| 80 | |
| 81 | **`skills/index.ts`** — import the new file and add it to `bundledSkills`: |
| 82 | |
| 83 | ```typescript |
| 84 | import myRunbook from "./my-runbook.md"; |
| 85 | |
| 86 | export const bundledSkills: Record<RunbookName, string> = { |
| 87 | // ... existing runbooks ... |
| 88 | "my-runbook": myRunbook, |
| 89 | }; |
| 90 | ``` |
| 91 | |
| 92 | The key must exactly match the `name` field in the runbook's YAML frontmatter. |
| 93 | |
| 94 | **`config.ts`** — assign the runbook to a mode: |
| 95 | |
| 96 | ```typescript |
| 97 | runbooksDaily: [ |
| 98 | "existing-runbook", |
| 99 | "my-runbook", // add here for daily checks |
| 100 | ], |
| 101 | runbooksWeekly: [ |
| 102 | "aws-cost-analysis", // or here for weekly checks |
| 103 | ], |
| 104 | ``` |
| 105 | |
| 106 | **Choose the mode:** |
| 107 | |
| 108 | - **`daily`** — checks that catch live regressions or fast-moving metrics |
| 109 | (error rates, queue depth, send compliance, log anomalies). |
| 110 | - **`weekly`** — checks where daily noise would exceed signal (cost trends, |
| 111 | coverage drift, long-horizon metrics). |
| 112 | |
| 113 | A runbook not in the index never runs. A runbook not assigned to a mode is |
| 114 | imported but never scheduled. |
| 115 | |
| 116 | ### 5. Remind the user |
| 117 | |
| 118 | Tell the user: **the runbook is inert until the doctor is redeployed.** Point |
| 119 | them to the doctor's deploy instructions (typically `sam deploy` or the |
| 120 | doctor's CI pipeline). The new runbook will not execute in the next scheduled |
| 121 | audit cycle until the updated build is live. |
| 122 | |
| 123 | --- |
| 124 | |
| 125 | ## Authoring checklist (condensed) |
| 126 | |
| 127 | Full checklist is in |
| 128 | [`../scaffold-doctor/references/writing-runbooks.md`](../scaffold-doctor/references/writing-runbooks.md). |
| 129 | |
| 130 | - [ ] Runbook file cre |