troubleshooting

Name: troubleshooting
Author: chaterm

50installs12forks

How to install troubleshooting?

chaterm/terminal-skills/troubleshooting

$npx -y skills add chaterm/terminal-skills --skill troubleshooting

Installs into the current project.

Summary

Kubernetes 故障排查

Files · 1

View on GitHub

SKILL.md

1	# Kubernetes 故障排查
2
3	## 概述
4	故障诊断、事件分析、资源调试等技能。
5
6	## 集群状态检查
7
8	### 节点状态
9	```bash
10	# 查看节点
11	kubectl get nodes
12	kubectl get nodes -o wide
13
14	# 节点详情
15	kubectl describe node node-name
16
17	# 节点资源使用
18	kubectl top nodes
19
20	# 节点条件
21	kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}'
22	```
23
24	### 组件状态
25	```bash
26	# 查看组件状态
27	kubectl get componentstatuses
28	kubectl get cs
29
30	# 查看系统 Pod
31	kubectl get pods -n kube-system
32
33	# API Server 健康检查
34	kubectl get --raw='/healthz'
35	kubectl get --raw='/readyz'
36	```
37
38	## Pod 故障排查
39
40	### Pod 状态分析
41	```bash
42	# 查看 Pod 状态
43	kubectl get pods -o wide
44	kubectl get pods --field-selector status.phase!=Running
45
46	# Pod 详情
47	kubectl describe pod pod-name
48
49	# 查看事件
50	kubectl get events --sort-by='.lastTimestamp'
51	kubectl get events --field-selector involvedObject.name=pod-name
52	```
53
54	### 常见 Pod 状态
55
56	#### Pending
57	```bash
58	# 原因：资源不足、调度问题、PVC 未绑定
59
60	# 排查步骤
61	kubectl describe pod pod-name \| grep -A 10 Events
62	kubectl describe pod pod-name \| grep -A 5 "Conditions"
63
64	# 检查节点资源
65	kubectl describe nodes \| grep -A 5 "Allocated resources"
66
67	# 检查 PVC
68	kubectl get pvc
69	kubectl describe pvc pvc-name
70	```
71
72	#### ImagePullBackOff
73	```bash
74	# 原因：镜像不存在、认证失败、网络问题
75
76	# 排查步骤
77	kubectl describe pod pod-name \| grep -A 5 "Events"
78
79	# 检查镜像名
80	kubectl get pod pod-name -o jsonpath='{.spec.containers[*].image}'
81
82	# 检查 imagePullSecrets
83	kubectl get pod pod-name -o jsonpath='{.spec.imagePullSecrets}'
84
85	# 手动拉取测试
86	docker pull image-name
87	```
88
89	#### CrashLoopBackOff
90	```bash
91	# 原因：应用崩溃、配置错误、资源不足
92
93	# 查看日志
94	kubectl logs pod-name
95	kubectl logs pod-name --previous
96
97	# 查看退出码
98	kubectl describe pod pod-name \| grep -A 5 "Last State"
99
100	# 检查资源限制
101	kubectl describe pod pod-name \| grep -A 10 "Limits"
102
103	# 进入容器调试
104	kubectl exec -it pod-name -- sh
105	```
106
107	#### OOMKilled
108	```bash
109	# 原因：内存超限
110
111	# 查看退出原因
112	kubectl describe pod pod-name \| grep -i oom
113
114	# 查看资源使用
115	kubectl top pod pod-name
116
117	# 增加内存限制
118	kubectl set resources deployment/deploy-name --limits=memory=512Mi
119	```
120
121	## 网络故障排查
122
123	### Service 连通性
124	```bash
125	# 检查 Service
126	kubectl get svc service-name
127	kubectl describe svc service-name
128
129	# 检查 Endpoints
130	kubectl get endpoints service-name
131
132	# 测试连通性
133	kubectl run test --rm -it --image=busybox -- sh
134	wget -qO- http://service-name:port
135	nslookup service-name
136	```
137
138	### DNS 排查
139	```bash
140	# 检查 CoreDNS
141	kubectl get pods -n kube-system -l k8s-app=kube-dns
142	kubectl logs -n kube-system -l k8s-app=kube-dns
143
144	# DNS 测试
145	kubectl run test --rm -it --image=busybox -- nslookup kubernetes
146	kubectl run test --rm -it --image=busybox -- nslookup service-name.namespace.svc.cluster.local
147	```
148
149	### 网络策略
150	```bash
151	# 查看网络策略
152	kubectl get networkpolicy
153	kubectl describe networkpolicy policy-name
154
155	# 测试网络连通
156	kubectl run test --rm -it --image=nicolaka/netshoot -- bash
157	curl -v http://service-name:port
158	```
159
160	## 存储故障排查
161
162	### PV/PVC 问题
163	```bash
164	# 查看 PV/PVC
165	kubectl get pv
166	kubectl get pvc
167	kubectl describe pvc pvc-name
168
169	# 常见问题
170	# - PVC Pending: 没有匹配的 PV
171	# - PV Released: 需要手动回收或删除
172
173	# 检查 StorageClass
174	kubectl get storageclass
175	kubectl describe storageclass sc-name
176	```
177
178	### 挂载问题
179	```bash
180	# 查看挂载
181	kubectl describe pod pod-name \| grep -A 10 "Volumes"
182
183	# 检查 Pod 内挂载
184	kubectl exec pod-name -- df -h
185	kubectl exec pod-name -- ls -la /mount/path
186	```
187
188	## 资源问题排查
189
190	### 资源配额
191	```bash
192	# 查看配额
193	kubectl get resourcequota -n namespace
194	kubectl describe resourcequota quota-name
195
196	# 查看 LimitRange
197	kubectl get limitrange -n namespace
198	kubectl describe limitrange limit-name
199	```
200
201	### 资源使用
202	```bash
203	# Pod 资源使用
204	kubectl top pods
205	kubectl top pods --containers
206
207	# 节点资源使用
208	kubectl top nodes
209
210	# 资源请求/限制
211	kubectl describe pod pod-name \| grep -A 10 "Requests"
212	```
213
214	## 日志收集
215
216	### 容器日志
217	```bash
218	# 当前日志
219	kubectl logs pod-name
220	kubectl logs pod-name -c container-name
221
222	# 历史日志
223	kubectl logs pod-name --previous
224
225	# 实时日志
226	kubectl logs -f pod-name --since=1h
227
228	# 多 Pod 日志
229	kubectl logs -l app=nginx --all-containers
230	```
231
232	### 系统日志
233	```bash
234	# kubelet 日志
235	journalctl -u kubelet -f
236
237	# 容器运行时日志
238	journalctl -u containerd -f
239	journalctl -u docker -f
240
241	# API Server 日志
242	kubectl logs -n kube-system kube-apiserver-master
243	```
244
245	## 调试工具
246
247	### kubectl debug
248	```bash
249	# 调试 Pod
250	kubectl debug pod-name -it --image=busybox --target=container-name
251
252	# 复制 Pod 调试
253	kubectl debug pod-name -it --copy-to=debug-pod --container=debug --image=nicolaka/netshoot
254
255	# 节点调试
256	kubectl debug node/node-name -it --image=busybox
257	```
258
259	### 临时调试 Pod
260	```bash
261	# 网络调试
262	kubectl run netshoot --rm -it --image=nicolaka/netshoot -- bash
263
264	# DNS 调试
265	kubectl run dnsutils --rm -it --image=gcr.io/kubernetes-e2e-test-images/dnsutils -- sh
266
267	# 通用调试
268	kubectl run debug --rm -it --image=busybox -- sh
269	```
270
271	## 常见场景
272
273	### 场景 1：应用无法访问
274	```bash
275	# 1. 检查 Pod 状态
276	kubectl get pods -l app=myapp
277
278	# 2. 检查 Service
279	kubectl get svc myapp-service
280	kubectl get endpoints myapp-service
281
282	# 3. 检查 Ingress
283	kubectl get ingress
284	kubectl describe ingress myapp-ingress
285
286	# 4. 测试连通性
287	kubectl run test --rm -it --image=busybox -- wget -qO- http://myapp-service
288	```
289
290	### 场景 2：Pod 调度失败
291	```bash
292	# 1

People who liked this also viewed

6 picks

Type

azure-diagnosticsDebug Azure production issues on Azure using AppLens, Azure Monitor, resource health, and safe triage. WHEN: debug production issues, troubleshoot app service, app service high CPU, app service deploySkillsJul 2026472k1.3k
systematic-debuggingUse when encountering any bug, test failure, or unexpected behavior, before proposing fixesSkillsJul 2026197k259k
safe-debugRigor Debug / Rigor Audit skill for deep learning research work. Use when the user pastes a traceback, terminal error, CUDA OOM, checkpoint load failure, shape mismatch, NaN loss symptom, or training SkillsMay 2026161k507
explore-runRigor Improve / Rigor Explore run leaf skill for bounded exploratory evidence in deep learning research repositories. Use when the researcher explicitly authorizes exploratory runs such as small-subseSkillsMay 2026161k507
minimal-run-and-auditRigor Run skill for README-first deep learning repo reproduction. Use when the task is specifically to capture or normalize evidence from the selected smoke test or documented inference or evaluation SkillsMay 2026161k507
insforge-debugUse when diagnosing problems in an InsForge project — reactive failures (SDK error object, HTTP 4xx/5xx, gateway timeout 502/503/504, edge function failure or timeout, login/OAuth/auth errors, RLS denSkillsJul 202618k31

troubleshooting

50installs12forks

View on GitHub

How to install troubleshooting?

chaterm/terminal-skills/troubleshooting

$npx -y skills add chaterm/terminal-skills --skill troubleshooting

Installs into the current project.

Summary

Kubernetes 故障排查

Files · 1

View on GitHub

SKILL.md

1	# Kubernetes 故障排查
2
3	## 概述
4	故障诊断、事件分析、资源调试等技能。
5
6	## 集群状态检查
7
8	### 节点状态
9	```bash
10	# 查看节点
11	kubectl get nodes
12	kubectl get nodes -o wide
13
14	# 节点详情
15	kubectl describe node node-name
16
17	# 节点资源使用
18	kubectl top nodes
19
20	# 节点条件
21	kubectl get nodes -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}'
22	```
23
24	### 组件状态
25	```bash
26	# 查看组件状态
27	kubectl get componentstatuses
28	kubectl get cs
29
30	# 查看系统 Pod
31	kubectl get pods -n kube-system
32
33	# API Server 健康检查
34	kubectl get --raw='/healthz'
35	kubectl get --raw='/readyz'
36	```
37
38	## Pod 故障排查
39
40	### Pod 状态分析
41	```bash
42	# 查看 Pod 状态
43	kubectl get pods -o wide
44	kubectl get pods --field-selector status.phase!=Running
45
46	# Pod 详情
47	kubectl describe pod pod-name
48
49	# 查看事件
50	kubectl get events --sort-by='.lastTimestamp'
51	kubectl get events --field-selector involvedObject.name=pod-name
52	```
53
54	### 常见 Pod 状态
55
56	#### Pending
57	```bash
58	# 原因：资源不足、调度问题、PVC 未绑定
59
60	# 排查步骤
61	kubectl describe pod pod-name \| grep -A 10 Events
62	kubectl describe pod pod-name \| grep -A 5 "Conditions"
63
64	# 检查节点资源
65	kubectl describe nodes \| grep -A 5 "Allocated resources"
66
67	# 检查 PVC
68	kubectl get pvc
69	kubectl describe pvc pvc-name
70	```
71
72	#### ImagePullBackOff
73	```bash
74	# 原因：镜像不存在、认证失败、网络问题
75
76	# 排查步骤
77	kubectl describe pod pod-name \| grep -A 5 "Events"
78
79	# 检查镜像名
80	kubectl get pod pod-name -o jsonpath='{.spec.containers[*].image}'
81
82	# 检查 imagePullSecrets
83	kubectl get pod pod-name -o jsonpath='{.spec.imagePullSecrets}'
84
85	# 手动拉取测试
86	docker pull image-name
87	```
88
89	#### CrashLoopBackOff
90	```bash
91	# 原因：应用崩溃、配置错误、资源不足
92
93	# 查看日志
94	kubectl logs pod-name
95	kubectl logs pod-name --previous
96
97	# 查看退出码
98	kubectl describe pod pod-name \| grep -A 5 "Last State"
99
100	# 检查资源限制
101	kubectl describe pod pod-name \| grep -A 10 "Limits"
102
103	# 进入容器调试
104	kubectl exec -it pod-name -- sh
105	```
106
107	#### OOMKilled
108	```bash
109	# 原因：内存超限
110
111	# 查看退出原因
112	kubectl describe pod pod-name \| grep -i oom
113
114	# 查看资源使用
115	kubectl top pod pod-name
116
117	# 增加内存限制
118	kubectl set resources deployment/deploy-name --limits=memory=512Mi
119	```
120
121	## 网络故障排查
122
123	### Service 连通性
124	```bash
125	# 检查 Service
126	kubectl get svc service-name
127	kubectl describe svc service-name
128
129	# 检查 Endpoints
130	kubectl get endpoints service-name
131
132	# 测试连通性
133	kubectl run test --rm -it --image=busybox -- sh
134	wget -qO- http://service-name:port
135	nslookup service-name
136	```
137
138	### DNS 排查
139	```bash
140	# 检查 CoreDNS
141	kubectl get pods -n kube-system -l k8s-app=kube-dns
142	kubectl logs -n kube-system -l k8s-app=kube-dns
143
144	# DNS 测试
145	kubectl run test --rm -it --image=busybox -- nslookup kubernetes
146	kubectl run test --rm -it --image=busybox -- nslookup service-name.namespace.svc.cluster.local
147	```
148
149	### 网络策略
150	```bash
151	# 查看网络策略
152	kubectl get networkpolicy
153	kubectl describe networkpolicy policy-name
154
155	# 测试网络连通
156	kubectl run test --rm -it --image=nicolaka/netshoot -- bash
157	curl -v http://service-name:port
158	```
159
160	## 存储故障排查
161
162	### PV/PVC 问题
163	```bash
164	# 查看 PV/PVC
165	kubectl get pv
166	kubectl get pvc
167	kubectl describe pvc pvc-name
168
169	# 常见问题
170	# - PVC Pending: 没有匹配的 PV
171	# - PV Released: 需要手动回收或删除
172
173	# 检查 StorageClass
174	kubectl get storageclass
175	kubectl describe storageclass sc-name
176	```
177
178	### 挂载问题
179	```bash
180	# 查看挂载
181	kubectl describe pod pod-name \| grep -A 10 "Volumes"
182
183	# 检查 Pod 内挂载
184	kubectl exec pod-name -- df -h
185	kubectl exec pod-name -- ls -la /mount/path
186	```
187
188	## 资源问题排查
189
190	### 资源配额
191	```bash
192	# 查看配额
193	kubectl get resourcequota -n namespace
194	kubectl describe resourcequota quota-name
195
196	# 查看 LimitRange
197	kubectl get limitrange -n namespace
198	kubectl describe limitrange limit-name
199	```
200
201	### 资源使用
202	```bash
203	# Pod 资源使用
204	kubectl top pods
205	kubectl top pods --containers
206
207	# 节点资源使用
208	kubectl top nodes
209
210	# 资源请求/限制
211	kubectl describe pod pod-name \| grep -A 10 "Requests"
212	```
213
214	## 日志收集
215
216	### 容器日志
217	```bash
218	# 当前日志
219	kubectl logs pod-name
220	kubectl logs pod-name -c container-name
221
222	# 历史日志
223	kubectl logs pod-name --previous
224
225	# 实时日志
226	kubectl logs -f pod-name --since=1h
227
228	# 多 Pod 日志
229	kubectl logs -l app=nginx --all-containers
230	```
231
232	### 系统日志
233	```bash
234	# kubelet 日志
235	journalctl -u kubelet -f
236
237	# 容器运行时日志
238	journalctl -u containerd -f
239	journalctl -u docker -f
240
241	# API Server 日志
242	kubectl logs -n kube-system kube-apiserver-master
243	```
244
245	## 调试工具
246
247	### kubectl debug
248	```bash
249	# 调试 Pod
250	kubectl debug pod-name -it --image=busybox --target=container-name
251
252	# 复制 Pod 调试
253	kubectl debug pod-name -it --copy-to=debug-pod --container=debug --image=nicolaka/netshoot
254
255	# 节点调试
256	kubectl debug node/node-name -it --image=busybox
257	```
258
259	### 临时调试 Pod
260	```bash
261	# 网络调试
262	kubectl run netshoot --rm -it --image=nicolaka/netshoot -- bash
263
264	# DNS 调试
265	kubectl run dnsutils --rm -it --image=gcr.io/kubernetes-e2e-test-images/dnsutils -- sh
266
267	# 通用调试
268	kubectl run debug --rm -it --image=busybox -- sh
269	```
270
271	## 常见场景
272
273	### 场景 1：应用无法访问
274	```bash
275	# 1. 检查 Pod 状态
276	kubectl get pods -l app=myapp
277
278	# 2. 检查 Service
279	kubectl get svc myapp-service
280	kubectl get endpoints myapp-service
281
282	# 3. 检查 Ingress
283	kubectl get ingress
284	kubectl describe ingress myapp-ingress
285
286	# 4. 测试连通性
287	kubectl run test --rm -it --image=busybox -- wget -qO- http://myapp-service
288	```
289
290	### 场景 2：Pod 调度失败
291	```bash
292	# 1

People who liked this also viewed

6 picks

Type