managing-incidents

Name: managing-incidents
Author: ancoleman

45installs59forks

How to install managing-incidents?

ancoleman/ai-design-components/managing-incidents

$npx -y skills add ancoleman/ai-design-components --skill managing-incidents

Installs into the current project.

Summary

Guide incident response from detection to post-mortem using SRE principles, severity classification, on-call management, blameless culture, and communication protocols. Use when setting up incident processes, designing escalation policies, or conducting post-mortems.

Files · 1

View on GitHub

SKILL.md

1	# Incident Management
2
3	Provide end-to-end incident management guidance covering detection, response, communication, and learning. Emphasizes SRE culture, blameless post-mortems, and structured processes for high-reliability operations.
4
5	## When to Use This Skill
6
7	Apply this skill when:
8	- Setting up incident response processes for a team
9	- Designing on-call rotations and escalation policies
10	- Creating runbooks for common failure scenarios
11	- Conducting blameless post-mortems after incidents
12	- Implementing incident communication protocols (internal and external)
13	- Choosing incident management tooling and platforms
14	- Improving MTTR and incident frequency metrics
15
16	## Core Principles
17
18	### Incident Management Philosophy
19
20	Declare Early and Often: Do not wait for certainty. Declaring an incident enables coordination, can be downgraded if needed, and prevents delayed response.
21
22	Mitigation First, Root Cause Later: Stop customer impact immediately (rollback, disable feature, failover). Debug and fix root cause after stability restored.
23
24	Blameless Culture: Assume good intentions. Focus on how systems failed, not who failed. Create psychological safety for honest learning.
25
26	Clear Command Structure: Assign Incident Commander (IC) to own coordination. IC delegates tasks but does not do hands-on debugging.
27
28	Communication is Critical: Internal coordination via dedicated channels, external transparency via status pages. Update stakeholders every 15-30 minutes during critical incidents.
29
30	## Severity Classification
31
32	Standard severity levels with response times:
33
34	SEV0 (P0) - Critical Outage:
35	- Impact: Complete service outage, critical data loss, payment processing down
36	- Response: Page immediately 24/7, all hands on deck, executive notification
37	- Example: API completely down, entire customer base affected
38
39	SEV1 (P1) - Major Degradation:
40	- Impact: Major functionality degraded, significant customer subset affected
41	- Response: Page during business hours, escalate off-hours, IC assigned
42	- Example: 15% error rate, critical feature unavailable
43
44	SEV2 (P2) - Minor Issues:
45	- Impact: Minor functionality impaired, edge case bug, small user subset
46	- Response: Email/Slack alert, next business day response
47	- Example: UI glitch, non-critical feature slow
48
49	SEV3 (P3) - Low Impact:
50	- Impact: Cosmetic issues, no customer functionality affected
51	- Response: Ticket queue, planned sprint
52	- Example: Visual inconsistency, documentation error
53
54	For detailed severity decision framework and interactive classifier, see `references/severity-classification.md`.
55
56	## Incident Roles
57
58	Incident Commander (IC):
59	- Owns overall incident response and coordination
60	- Makes strategic decisions (rollback vs. debug, when to escalate)
61	- Delegates tasks to responders (does NOT do hands-on debugging)
62	- Declares incident resolved when stability confirmed
63
64	Communications Lead:
65	- Posts status updates to internal and external channels
66	- Coordinates with stakeholders (executives, product, support)
67	- Drafts post-incident customer communication
68	- Cadence: Every 15-30 minutes for SEV0/SEV1
69
70	Subject Matter Experts (SMEs):
71	- Hands-on debugging and mitigation
72	- Execute runbooks and implement fixes
73	- Provide technical context to IC
74
75	Scribe:
76	- Documents timeline, actions, decisions in real-time
77	- Records incident notes for post-mortem reconstruction
78
79	Assign roles based on severity:
80	- SEV2/SEV3: Single responder
81	- SEV1: IC + SME(s)
82	- SEV0: IC + Communications Lead + SME(s) + Scribe
83
84	For detailed role responsibilities, see `references/incident-roles.md`.
85
86	## On-Call Management
87
88	### Rotation Patterns
89
90	Primary + Secondary:
91	- Primary: First responder
92	- Secondary: Backup if primary doesn't ack within 5 minutes
93	- Rotation length: 1 week (optimal balance)
94
95	Follow-the-Sun (24/7):
96	- Team A: US hours, Team B: Europe hours, Team C: Asia hours
97	- Benefit: No night shifts, improved work-life balance
98	- Requires: Multiple global teams
99
100	Tiered Escalation:
101	- Tier 1: Junior on-call (common issues, runbook-driven)
102	- Tier 2: Senior on-call (complex troubleshooting)
103	- Tier 3: Team lead/architect (critical decisions)
104
105	### Best Practices
106
107	- Rotation length: 1 week per rotation
108	- Handoff ceremony: 30-minute call to discuss active issues
109	- Compensation: On-call stipend + time off after major incidents
110	- Tooling: PagerDuty, Opsgenie, or incident.io
111	- Limits: Max 2-3 pages per night; escalate if exceeded
112
113	## Incident Response Workflow
114
115	Standard incident lifecycle:
116
117	```
118	Detection → Triage → Declaration → Investigation
119	↓
120	Mitigation → Resolution → Monitoring → Closure
121	↓
122	Post-Mortem (within 48 hours)
123	```
124
125	### Key Decision Points
126
127	**When to Declar

People who liked this also viewed

6 picks

Type

managing-incidents

45installs59forks

View on GitHub

How to install managing-incidents?

ancoleman/ai-design-components/managing-incidents

$npx -y skills add ancoleman/ai-design-components --skill managing-incidents

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Incident Management
2
3	Provide end-to-end incident management guidance covering detection, response, communication, and learning. Emphasizes SRE culture, blameless post-mortems, and structured processes for high-reliability operations.
4
5	## When to Use This Skill
6
7	Apply this skill when:
8	- Setting up incident response processes for a team
9	- Designing on-call rotations and escalation policies
10	- Creating runbooks for common failure scenarios
11	- Conducting blameless post-mortems after incidents
12	- Implementing incident communication protocols (internal and external)
13	- Choosing incident management tooling and platforms
14	- Improving MTTR and incident frequency metrics
15
16	## Core Principles
17
18	### Incident Management Philosophy
19
20	Declare Early and Often: Do not wait for certainty. Declaring an incident enables coordination, can be downgraded if needed, and prevents delayed response.
21
22	Mitigation First, Root Cause Later: Stop customer impact immediately (rollback, disable feature, failover). Debug and fix root cause after stability restored.
23
24	Blameless Culture: Assume good intentions. Focus on how systems failed, not who failed. Create psychological safety for honest learning.
25
26	Clear Command Structure: Assign Incident Commander (IC) to own coordination. IC delegates tasks but does not do hands-on debugging.
27
28	Communication is Critical: Internal coordination via dedicated channels, external transparency via status pages. Update stakeholders every 15-30 minutes during critical incidents.
29
30	## Severity Classification
31
32	Standard severity levels with response times:
33
34	SEV0 (P0) - Critical Outage:
35	- Impact: Complete service outage, critical data loss, payment processing down
36	- Response: Page immediately 24/7, all hands on deck, executive notification
37	- Example: API completely down, entire customer base affected
38
39	SEV1 (P1) - Major Degradation:
40	- Impact: Major functionality degraded, significant customer subset affected
41	- Response: Page during business hours, escalate off-hours, IC assigned
42	- Example: 15% error rate, critical feature unavailable
43
44	SEV2 (P2) - Minor Issues:
45	- Impact: Minor functionality impaired, edge case bug, small user subset
46	- Response: Email/Slack alert, next business day response
47	- Example: UI glitch, non-critical feature slow
48
49	SEV3 (P3) - Low Impact:
50	- Impact: Cosmetic issues, no customer functionality affected
51	- Response: Ticket queue, planned sprint
52	- Example: Visual inconsistency, documentation error
53
54	For detailed severity decision framework and interactive classifier, see `references/severity-classification.md`.
55
56	## Incident Roles
57
58	Incident Commander (IC):
59	- Owns overall incident response and coordination
60	- Makes strategic decisions (rollback vs. debug, when to escalate)
61	- Delegates tasks to responders (does NOT do hands-on debugging)
62	- Declares incident resolved when stability confirmed
63
64	Communications Lead:
65	- Posts status updates to internal and external channels
66	- Coordinates with stakeholders (executives, product, support)
67	- Drafts post-incident customer communication
68	- Cadence: Every 15-30 minutes for SEV0/SEV1
69
70	Subject Matter Experts (SMEs):
71	- Hands-on debugging and mitigation
72	- Execute runbooks and implement fixes
73	- Provide technical context to IC
74
75	Scribe:
76	- Documents timeline, actions, decisions in real-time
77	- Records incident notes for post-mortem reconstruction
78
79	Assign roles based on severity:
80	- SEV2/SEV3: Single responder
81	- SEV1: IC + SME(s)
82	- SEV0: IC + Communications Lead + SME(s) + Scribe
83
84	For detailed role responsibilities, see `references/incident-roles.md`.
85
86	## On-Call Management
87
88	### Rotation Patterns
89
90	Primary + Secondary:
91	- Primary: First responder
92	- Secondary: Backup if primary doesn't ack within 5 minutes
93	- Rotation length: 1 week (optimal balance)
94
95	Follow-the-Sun (24/7):
96	- Team A: US hours, Team B: Europe hours, Team C: Asia hours
97	- Benefit: No night shifts, improved work-life balance
98	- Requires: Multiple global teams
99
100	Tiered Escalation:
101	- Tier 1: Junior on-call (common issues, runbook-driven)
102	- Tier 2: Senior on-call (complex troubleshooting)
103	- Tier 3: Team lead/architect (critical decisions)
104
105	### Best Practices
106
107	- Rotation length: 1 week per rotation
108	- Handoff ceremony: 30-minute call to discuss active issues
109	- Compensation: On-call stipend + time off after major incidents
110	- Tooling: PagerDuty, Opsgenie, or incident.io
111	- Limits: Max 2-3 pages per night; escalate if exceeded
112
113	## Incident Response Workflow
114
115	Standard incident lifecycle:
116
117	```
118	Detection → Triage → Declaration → Investigation
119	↓
120	Mitigation → Resolution → Monitoring → Closure
121	↓
122	Post-Mortem (within 48 hours)
123	```
124
125	### Key Decision Points
126
127	**When to Declar

People who liked this also viewed

6 picks

Type