aks-deployment-troubleshooter

Name: aks-deployment-troubleshooter
Author: mjunaidca

26stars11forks

How to install aks-deployment-troubleshooter?

mjunaidca/mjs-agent-skills/aks-deployment-troubleshooter

$npx -y skills add mjunaidca/mjs-agent-skills --skill aks-deployment-troubleshooter

Installs into the current project.

Summary

Diagnose and fix Kubernetes deployment failures, especially ImagePullBackOff, CrashLoopBackOff, and architecture mismatches. Battle-tested from 4-hour AKS debugging session with 10+ failure modes resolved.

Files · 1

View on GitHub

SKILL.md

1	# AKS Deployment Troubleshooter
2
3	## Overview
4
5	This skill captures systematic approaches to debugging Kubernetes deployments, with specific focus on container image issues. Based on real debugging session resolving 10+ different failure modes.
6
7	## When to Use
8
9	- Pods stuck in `ImagePullBackOff`
10	- Pods in `CrashLoopBackOff` with `exec format error`
11	- "no match for platform in manifest" errors
12	- Image registry authentication issues
13	- Helm deployment timeouts
14
15	## Quick Diagnosis Flow
16
17	```
18	Pod not running?
19	│
20	├─► ImagePullBackOff
21	│ │
22	│ ├─► "not found" ──► Wrong tag or registry path
23	│ ├─► "unauthorized" ──► Auth/imagePullSecrets issue
24	│ └─► "no match for platform" ──► Architecture mismatch
25	│
26	├─► CrashLoopBackOff
27	│ │
28	│ ├─► "exec format error" ──► Wrong CPU architecture
29	│ ├─► Exit code 1 ──► App startup failure (check logs)
30	│ └─► OOMKilled ──► Memory limits too low
31	│
32	└─► Pending
33	│
34	├─► Insufficient CPU/memory ──► Scale cluster or reduce requests
35	└─► No matching node ──► Check nodeSelector/tolerations
36	```
37
38	## Diagnostic Commands
39
40	### Step 1: Get Pod Status
41	```bash
42	kubectl get pods -n <namespace>
43	```
44
45	### Step 2: Describe Failing Pod
46	```bash
47	kubectl describe pod <pod-name> -n <namespace> \| grep -E "(Image:\|Failed\|Error\|pull)"
48	```
49
50	### Step 3: Check Events
51	```bash
52	kubectl get events -n <namespace> --sort-by='.lastTimestamp' \| tail -20
53	```
54
55	### Step 4: Check Logs (for CrashLoopBackOff)
56	```bash
57	kubectl logs <pod-name> -n <namespace> --tail=50
58	```
59
60	### Step 5: Check Node Architecture
61	```bash
62	kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.architecture}'
63	```
64
65	## Error Resolution Guide
66
67	### 1. ImagePullBackOff: "not found"
68
69	Error:
70	```
71	Failed to pull image "ghcr.io/owner/repo/app:abc123": not found
72	```
73
74	Causes & Solutions:
75
76	\| Cause \| Solution \|
77	\|-------\|----------\|
78	\| Tag doesn't exist \| Verify image was pushed with exact tag \|
79	\| Short vs full SHA \| Align metadata-action with deploy (use `type=raw,value=${{ github.sha }}`) \|
80	\| Builds skipped \| Manual trigger or remove path filters \|
81	\| Wrong registry \| Check `image.repository` in Helm values \|
82
83	Diagnostic:
84	```bash
85	# Check what tags exist (requires gh cli and package visibility)
86	gh api /users/<owner>/packages/container/<package>/versions
87	```
88
89	---
90
91	### 2. ImagePullBackOff: "unauthorized"
92
93	Error:
94	```
95	failed to authorize: failed to fetch anonymous token: 401 Unauthorized
96	```
97
98	Causes & Solutions:
99
100	\| Cause \| Solution \|
101	\|-------\|----------\|
102	\| Package is private \| Make package public in GHCR settings \|
103	\| Missing imagePullSecrets \| Create docker-registry secret \|
104	\| Wrong credentials \| Regenerate and update secret \|
105
106	Create imagePullSecrets:
107	```bash
108	kubectl create secret docker-registry ghcr-secret \
109	--docker-server=ghcr.io \
110	--docker-username=<github-username> \
111	--docker-password=<github-token> \
112	--namespace=<namespace>
113	```
114
115	Link secret in deployment:
116	```yaml
117	spec:
118	imagePullSecrets:
119	- name: ghcr-secret
120	```
121
122	---
123
124	### 3. ImagePullBackOff: "no match for platform in manifest"
125
126	Error:
127	```
128	no match for platform in manifest: not found
129	```
130
131	Root Cause: Image built for wrong CPU architecture OR buildx provenance issue.
132
133	Step 1: Check cluster architecture:
134	```bash
135	kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.architecture}'
136	# Output: amd64 amd64 OR arm64 arm64
137	```
138
139	Step 2: Match build platform:
140	```yaml
141	# In GitHub Actions docker/build-push-action
142	- uses: docker/build-push-action@v5
143	with:
144	platforms: linux/arm64 # or linux/amd64
145	provenance: false # CRITICAL: Disable attestation manifests
146	no-cache: true # Force fresh build
147	```
148
149	Why `provenance: false`?
150	Buildx creates multi-arch manifest lists with attestations. Some container runtimes can't find the actual image in complex manifests. Disabling provenance creates simple single-platform images.
151
152	---
153
154	### 4. CrashLoopBackOff: "exec format error"
155
156	Error:
157	```
158	exec /usr/local/bin/docker-entrypoint.sh: exec format error
159	```
160
161	Root Cause: Binary architecture doesn't match node architecture.
162
163	Example: Built `linux/amd64` image, deployed to `arm64` nodes.
164
165	Solution:
166	1. Check node architecture: `kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.architecture}'`
167	2. Update build platform to match
168	3. Rebuild WITHOUT cache (cached layers may have wrong arch)
169
170	```yaml
171	platforms: linux/arm64 # Match your cluster!
172	no-cache: true # Force complete rebuild
173	provenance: false # Simple manifest
174	```
175
176	---
177
178	### 5. Helm --set Comma Parsing Error
179
180	Error:
181	```
182	failed parsing --set data: key "com" has no value (can

People who liked this also viewed

6 picks

Type

aks-deployment-troubleshooter

26stars11forks

View on GitHub

How to install aks-deployment-troubleshooter?

mjunaidca/mjs-agent-skills/aks-deployment-troubleshooter

$npx -y skills add mjunaidca/mjs-agent-skills --skill aks-deployment-troubleshooter

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# AKS Deployment Troubleshooter
2
3	## Overview
4
5	This skill captures systematic approaches to debugging Kubernetes deployments, with specific focus on container image issues. Based on real debugging session resolving 10+ different failure modes.
6
7	## When to Use
8
9	- Pods stuck in `ImagePullBackOff`
10	- Pods in `CrashLoopBackOff` with `exec format error`
11	- "no match for platform in manifest" errors
12	- Image registry authentication issues
13	- Helm deployment timeouts
14
15	## Quick Diagnosis Flow
16
17	```
18	Pod not running?
19	│
20	├─► ImagePullBackOff
21	│ │
22	│ ├─► "not found" ──► Wrong tag or registry path
23	│ ├─► "unauthorized" ──► Auth/imagePullSecrets issue
24	│ └─► "no match for platform" ──► Architecture mismatch
25	│
26	├─► CrashLoopBackOff
27	│ │
28	│ ├─► "exec format error" ──► Wrong CPU architecture
29	│ ├─► Exit code 1 ──► App startup failure (check logs)
30	│ └─► OOMKilled ──► Memory limits too low
31	│
32	└─► Pending
33	│
34	├─► Insufficient CPU/memory ──► Scale cluster or reduce requests
35	└─► No matching node ──► Check nodeSelector/tolerations
36	```
37
38	## Diagnostic Commands
39
40	### Step 1: Get Pod Status
41	```bash
42	kubectl get pods -n <namespace>
43	```
44
45	### Step 2: Describe Failing Pod
46	```bash
47	kubectl describe pod <pod-name> -n <namespace> \| grep -E "(Image:\|Failed\|Error\|pull)"
48	```
49
50	### Step 3: Check Events
51	```bash
52	kubectl get events -n <namespace> --sort-by='.lastTimestamp' \| tail -20
53	```
54
55	### Step 4: Check Logs (for CrashLoopBackOff)
56	```bash
57	kubectl logs <pod-name> -n <namespace> --tail=50
58	```
59
60	### Step 5: Check Node Architecture
61	```bash
62	kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.architecture}'
63	```
64
65	## Error Resolution Guide
66
67	### 1. ImagePullBackOff: "not found"
68
69	Error:
70	```
71	Failed to pull image "ghcr.io/owner/repo/app:abc123": not found
72	```
73
74	Causes & Solutions:
75
76	\| Cause \| Solution \|
77	\|-------\|----------\|
78	\| Tag doesn't exist \| Verify image was pushed with exact tag \|
79	\| Short vs full SHA \| Align metadata-action with deploy (use `type=raw,value=${{ github.sha }}`) \|
80	\| Builds skipped \| Manual trigger or remove path filters \|
81	\| Wrong registry \| Check `image.repository` in Helm values \|
82
83	Diagnostic:
84	```bash
85	# Check what tags exist (requires gh cli and package visibility)
86	gh api /users/<owner>/packages/container/<package>/versions
87	```
88
89	---
90
91	### 2. ImagePullBackOff: "unauthorized"
92
93	Error:
94	```
95	failed to authorize: failed to fetch anonymous token: 401 Unauthorized
96	```
97
98	Causes & Solutions:
99
100	\| Cause \| Solution \|
101	\|-------\|----------\|
102	\| Package is private \| Make package public in GHCR settings \|
103	\| Missing imagePullSecrets \| Create docker-registry secret \|
104	\| Wrong credentials \| Regenerate and update secret \|
105
106	Create imagePullSecrets:
107	```bash
108	kubectl create secret docker-registry ghcr-secret \
109	--docker-server=ghcr.io \
110	--docker-username=<github-username> \
111	--docker-password=<github-token> \
112	--namespace=<namespace>
113	```
114
115	Link secret in deployment:
116	```yaml
117	spec:
118	imagePullSecrets:
119	- name: ghcr-secret
120	```
121
122	---
123
124	### 3. ImagePullBackOff: "no match for platform in manifest"
125
126	Error:
127	```
128	no match for platform in manifest: not found
129	```
130
131	Root Cause: Image built for wrong CPU architecture OR buildx provenance issue.
132
133	Step 1: Check cluster architecture:
134	```bash
135	kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.architecture}'
136	# Output: amd64 amd64 OR arm64 arm64
137	```
138
139	Step 2: Match build platform:
140	```yaml
141	# In GitHub Actions docker/build-push-action
142	- uses: docker/build-push-action@v5
143	with:
144	platforms: linux/arm64 # or linux/amd64
145	provenance: false # CRITICAL: Disable attestation manifests
146	no-cache: true # Force fresh build
147	```
148
149	Why `provenance: false`?
150	Buildx creates multi-arch manifest lists with attestations. Some container runtimes can't find the actual image in complex manifests. Disabling provenance creates simple single-platform images.
151
152	---
153
154	### 4. CrashLoopBackOff: "exec format error"
155
156	Error:
157	```
158	exec /usr/local/bin/docker-entrypoint.sh: exec format error
159	```
160
161	Root Cause: Binary architecture doesn't match node architecture.
162
163	Example: Built `linux/amd64` image, deployed to `arm64` nodes.
164
165	Solution:
166	1. Check node architecture: `kubectl get nodes -o jsonpath='{.items[*].status.nodeInfo.architecture}'`
167	2. Update build platform to match
168	3. Rebuild WITHOUT cache (cached layers may have wrong arch)
169
170	```yaml
171	platforms: linux/arm64 # Match your cluster!
172	no-cache: true # Force complete rebuild
173	provenance: false # Simple manifest
174	```
175
176	---
177
178	### 5. Helm --set Comma Parsing Error
179
180	Error:
181	```
182	failed parsing --set data: key "com" has no value (can

People who liked this also viewed

6 picks

Type