chaos-engineering

Name: chaos-engineering
Author: vibeeval

519stars44forks

How to install chaos-engineering?

vibeeval/vibecosystem/chaos-engineering

$npx -y skills add vibeeval/vibecosystem --skill chaos-engineering

Installs into the current project.

Summary

Failure injection patterns, blast radius control, steady state hypothesis, and gameday planning for resilience testing.

Files · 1

View on GitHub

SKILL.md

1	# Chaos Engineering
2
3	Systematic resilience testing to discover weaknesses before they cause outages.
4
5	## Steady State Hypothesis
6
7	```yaml
8	# Define BEFORE injecting chaos - what "normal" looks like
9	steady_state_hypothesis:
10	title: "API serves traffic within SLO"
11	probes:
12	- name: "API response time p95 < 500ms"
13	type: http
14	url: "https://api.example.com/health"
15	threshold: 500
16
17	- name: "Error rate < 1%"
18	type: prometheus
19	query: "rate(http_requests_total{status=~'5..'}[5m]) / rate(http_requests_total[5m])"
20	threshold: 0.01
21
22	- name: "Order processing queue depth < 100"
23	type: cloudwatch
24	metric: "ApproximateNumberOfMessagesVisible"
25	threshold: 100
26
27	- name: "Database connections < 80% capacity"
28	type: prometheus
29	query: "pg_stat_activity_count / pg_settings_max_connections"
30	threshold: 0.8
31	```
32
33	## Failure Injection Patterns
34
35	```python
36	# Using Chaos Toolkit (chaostoolkit.org)
37	# experiment.json
38
39	{
40	"title": "Database failover resilience",
41	"description": "Verify app handles primary DB failover gracefully",
42
43	"steady-state-hypothesis": {
44	"title": "API responds normally",
45	"probes": [
46	{
47	"name": "api-health",
48	"type": "probe",
49	"provider": {
50	"type": "http",
51	"url": "https://api.example.com/health",
52	"timeout": 5
53	},
54	"tolerance": {"status": 200}
55	}
56	]
57	},
58
59	"method": [
60	{
61	"name": "failover-primary-db",
62	"type": "action",
63	"provider": {
64	"type": "python",
65	"module": "chaosaws.rds.actions",
66	"func": "failover_db_cluster",
67	"arguments": {
68	"db_cluster_identifier": "prod-cluster"
69	}
70	},
71	"pauses": {"after": 60}
72	}
73	],
74
75	"rollbacks": [
76	{
77	"name": "verify-db-recovered",
78	"type": "probe",
79	"provider": {
80	"type": "python",
81	"module": "chaosaws.rds.probes",
82	"func": "cluster_status",
83	"arguments": {
84	"db_cluster_identifier": "prod-cluster"
85	}
86	},
87	"tolerance": "available"
88	}
89	]
90	}
91	```
92
93	## Blast Radius Control
94
95	```python
96	# ALWAYS limit the impact of chaos experiments
97
98	class BlastRadiusController:
99	"""Control and limit chaos experiment impact."""
100
101	def __init__(self, config: dict):
102	self.max_affected_percentage = config.get('max_affected_pct', 5)
103	self.max_duration_seconds = config.get('max_duration_s', 300)
104	self.excluded_services = config.get('excluded', ['auth', 'payments'])
105	self.kill_switch_url = config.get('kill_switch_url')
106
107	def can_inject(self, target: str, scope: str) -> bool:
108	# Never chaos-test critical services without explicit approval
109	if target in self.excluded_services:
110	return False
111
112	# Never inject during peak hours
113	hour = datetime.now().hour
114	if 9 <= hour <= 17: # Business hours (adjust per timezone)
115	return False
116
117	# Never affect more than N% of instances
118	if self.get_affected_percentage(target, scope) > self.max_affected_percentage:
119	return False
120
121	return True
122
123	def get_affected_percentage(self, target: str, scope: str) -> float:
124	total = self.get_total_instances(target)
125	affected = self.get_affected_instances(target, scope)
126	return (affected / total) * 100 if total > 0 else 100
127
128	async def emergency_stop(self) -> None:
129	"""Kill switch: immediately halt all chaos experiments."""
130	await httpx.post(self.kill_switch_url, json={"action": "stop_all"})
131	```
132
133	## Common Chaos Experiments
134
135	```yaml
136	# Experiment catalog - start with these
137
138	level_1_basic:
139	- name: "Kill a single pod"
140	tool: "kubectl delete pod <name>"
141	validates: "Pod auto-recovery, health checks"
142	blast_radius: "1 pod"
143
144	- name: "CPU stress on one node"
145	tool: "stress-ng --cpu 4 --timeout 60"
146	validates: "Autoscaling, request routing"
147	blast_radius: "1 node"
148
149	- name: "Inject 500ms network latency"
150	tool: "tc qdisc add dev eth0 root netem delay 500ms"
151	validates: "Timeout handling, circuit breakers"
152	blast_radius: "1 container"
153
154	level_2_intermediate:
155	- name: "Kill entire availability zone"
156	tool: "Chaos Toolkit / AWS FIS"
157	validates: "Multi-AZ failover, data replication"
158	blast_radius: "1 AZ"
159
160	- name: "DNS resolution failure"
161	tool: "iptables -A OUTPUT -p udp --dport 53 -j DROP"
162	validates: "DNS caching, fallback resolution"
163	blast_radius: "1 service"
164
165	- name: "Disk fill to 95%"
166	tool: "fallocate -l 50G /tmp/disk_fill"
167	validates: "Disk space alerts, log rotation"
168	blast_radius: "1 node"
169
170	level_3_advanced:
171	- name: "Split brain network partition"
172	tool: "Toxiproxy / Linux iptables"
173	validates: "Consensus protocols, data consistency"

People who liked this also viewed

6 picks

Type

tddSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026501k181k
test-driven-developmentUse when implementing any feature or bugfix, before writing implementation codeSkillsJul 2026175k259k
qaSkills for Real Engineers. Straight from my .agents directory.SkillsJul 2026160k181k
verification-before-completionUse when about to claim work is complete, fixed, or passing, before committing or creating PRs - requires running verification commands and confirming output before making any success claims; evidenceSkillsJul 2026153k259k
webapp-testingToolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser lSkillsJul 2026120k163k
playwright-best-practicesUse when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, teSkillsJul 202664k338

chaos-engineering

519stars44forks

View on GitHub

How to install chaos-engineering?

vibeeval/vibecosystem/chaos-engineering

$npx -y skills add vibeeval/vibecosystem --skill chaos-engineering

Installs into the current project.

Summary

Failure injection patterns, blast radius control, steady state hypothesis, and gameday planning for resilience testing.

Files · 1

View on GitHub

SKILL.md

1	# Chaos Engineering
2
3	Systematic resilience testing to discover weaknesses before they cause outages.
4
5	## Steady State Hypothesis
6
7	```yaml
8	# Define BEFORE injecting chaos - what "normal" looks like
9	steady_state_hypothesis:
10	title: "API serves traffic within SLO"
11	probes:
12	- name: "API response time p95 < 500ms"
13	type: http
14	url: "https://api.example.com/health"
15	threshold: 500
16
17	- name: "Error rate < 1%"
18	type: prometheus
19	query: "rate(http_requests_total{status=~'5..'}[5m]) / rate(http_requests_total[5m])"
20	threshold: 0.01
21
22	- name: "Order processing queue depth < 100"
23	type: cloudwatch
24	metric: "ApproximateNumberOfMessagesVisible"
25	threshold: 100
26
27	- name: "Database connections < 80% capacity"
28	type: prometheus
29	query: "pg_stat_activity_count / pg_settings_max_connections"
30	threshold: 0.8
31	```
32
33	## Failure Injection Patterns
34
35	```python
36	# Using Chaos Toolkit (chaostoolkit.org)
37	# experiment.json
38
39	{
40	"title": "Database failover resilience",
41	"description": "Verify app handles primary DB failover gracefully",
42
43	"steady-state-hypothesis": {
44	"title": "API responds normally",
45	"probes": [
46	{
47	"name": "api-health",
48	"type": "probe",
49	"provider": {
50	"type": "http",
51	"url": "https://api.example.com/health",
52	"timeout": 5
53	},
54	"tolerance": {"status": 200}
55	}
56	]
57	},
58
59	"method": [
60	{
61	"name": "failover-primary-db",
62	"type": "action",
63	"provider": {
64	"type": "python",
65	"module": "chaosaws.rds.actions",
66	"func": "failover_db_cluster",
67	"arguments": {
68	"db_cluster_identifier": "prod-cluster"
69	}
70	},
71	"pauses": {"after": 60}
72	}
73	],
74
75	"rollbacks": [
76	{
77	"name": "verify-db-recovered",
78	"type": "probe",
79	"provider": {
80	"type": "python",
81	"module": "chaosaws.rds.probes",
82	"func": "cluster_status",
83	"arguments": {
84	"db_cluster_identifier": "prod-cluster"
85	}
86	},
87	"tolerance": "available"
88	}
89	]
90	}
91	```
92
93	## Blast Radius Control
94
95	```python
96	# ALWAYS limit the impact of chaos experiments
97
98	class BlastRadiusController:
99	"""Control and limit chaos experiment impact."""
100
101	def __init__(self, config: dict):
102	self.max_affected_percentage = config.get('max_affected_pct', 5)
103	self.max_duration_seconds = config.get('max_duration_s', 300)
104	self.excluded_services = config.get('excluded', ['auth', 'payments'])
105	self.kill_switch_url = config.get('kill_switch_url')
106
107	def can_inject(self, target: str, scope: str) -> bool:
108	# Never chaos-test critical services without explicit approval
109	if target in self.excluded_services:
110	return False
111
112	# Never inject during peak hours
113	hour = datetime.now().hour
114	if 9 <= hour <= 17: # Business hours (adjust per timezone)
115	return False
116
117	# Never affect more than N% of instances
118	if self.get_affected_percentage(target, scope) > self.max_affected_percentage:
119	return False
120
121	return True
122
123	def get_affected_percentage(self, target: str, scope: str) -> float:
124	total = self.get_total_instances(target)
125	affected = self.get_affected_instances(target, scope)
126	return (affected / total) * 100 if total > 0 else 100
127
128	async def emergency_stop(self) -> None:
129	"""Kill switch: immediately halt all chaos experiments."""
130	await httpx.post(self.kill_switch_url, json={"action": "stop_all"})
131	```
132
133	## Common Chaos Experiments
134
135	```yaml
136	# Experiment catalog - start with these
137
138	level_1_basic:
139	- name: "Kill a single pod"
140	tool: "kubectl delete pod <name>"
141	validates: "Pod auto-recovery, health checks"
142	blast_radius: "1 pod"
143
144	- name: "CPU stress on one node"
145	tool: "stress-ng --cpu 4 --timeout 60"
146	validates: "Autoscaling, request routing"
147	blast_radius: "1 node"
148
149	- name: "Inject 500ms network latency"
150	tool: "tc qdisc add dev eth0 root netem delay 500ms"
151	validates: "Timeout handling, circuit breakers"
152	blast_radius: "1 container"
153
154	level_2_intermediate:
155	- name: "Kill entire availability zone"
156	tool: "Chaos Toolkit / AWS FIS"
157	validates: "Multi-AZ failover, data replication"
158	blast_radius: "1 AZ"
159
160	- name: "DNS resolution failure"
161	tool: "iptables -A OUTPUT -p udp --dport 53 -j DROP"
162	validates: "DNS caching, fallback resolution"
163	blast_radius: "1 service"
164
165	- name: "Disk fill to 95%"
166	tool: "fallocate -l 50G /tmp/disk_fill"
167	validates: "Disk space alerts, log rotation"
168	blast_radius: "1 node"
169
170	level_3_advanced:
171	- name: "Split brain network partition"
172	tool: "Toxiproxy / Linux iptables"
173	validates: "Consensus protocols, data consistency"

People who liked this also viewed

6 picks

Type