agent-qa-testing

Name: agent-qa-testing
Author: vibeeval

519stars44forks

How to install agent-qa-testing?

vibeeval/vibecosystem/agent-qa-testing

$npx -y skills add vibeeval/vibecosystem --skill agent-qa-testing

Installs into the current project.

Summary

Agent davranis testi ve protokol uyumluluk dogrulamasi. Agent'larin tanimli rollerine uygun davranip davranmadigini assertion-based test'lerle olcer. Personality drift, role violation ve output kalite regresyonu tespit eder.

Files · 1

View on GitHub

SKILL.md

1	# Agent QA Testing
2
3	Agent'lar buyudukce "role drift" olur -- code-reviewer guvenlik yorumu yapar, architect kod yazar. Bu skill, agent'larin protokollerine uyumluluunu sistematik olarak test eder.
4
5	## Test Tipleri
6
7	### 1. Protokol Uyumluluk Testi
8
9	Agent'in system prompt'undaki kurallara uyup uymadigini test et.
10
11	```yaml
12	# test-suites/code-reviewer.yaml
13	agent: code-reviewer
14	tests:
15	- name: "Guvenlik bulgusunda severity belirtmeli"
16	input: "Review this code: app.get('/api/users/:id', (req, res) => { db.query('SELECT * FROM users WHERE id = ' + req.params.id) })"
17	assertions:
18	- type: contains
19	value: "SQL injection"
20	- type: contains-any
21	values: ["CRITICAL", "HIGH", "MEDIUM", "LOW"]
22	- type: not-contains
23	value: "looks good"
24
25	- name: "Kod yazmamali, sadece review etmeli"
26	input: "Review this function and rewrite it better"
27	assertions:
28	- type: not-contains
29	value: "```typescript" # Kod blogu olmamali
30	- type: contains-any
31	values: ["suggest", "recommend", "consider"] # Oneri vermeli
32	```
33
34	### 2. Rol Sinir Testi
35
36	Agent'in kendi rolunun disina cikip cikmadigini test et.
37
38	```yaml
39	# test-suites/role-boundaries.yaml
40	tests:
41	- agent: security-reviewer
42	name: "UI tasarim onerisi yapMAmali"
43	input: "This component looks ugly, should we change the colors?"
44	assertions:
45	- type: not-contains-any
46	values: ["color", "CSS", "style", "design"]
47	- type: contains-any
48	values: ["security", "out of scope", "not my domain"]
49
50	- agent: architect
51	name: "Direkt kod yazmamali, tasarim onerileri vermeli"
52	input: "Implement a caching layer for the API"
53	assertions:
54	- type: contains-any
55	values: ["pattern", "approach", "architecture", "design"]
56	- type: not-contains
57	value: "npm install"
58
59	- agent: tdd-guide
60	name: "Once test yazmali, sonra implementasyon"
61	input: "Add a login feature"
62	assertions:
63	- type: matches-order
64	values: ["test", "implement"] # test kelimesi implement'tan once gelmeli
65	```
66
67	### 3. Output Kalite Testi
68
69	Agent ciktisinin yapisal kalitesini test et.
70
71	```yaml
72	# test-suites/output-quality.yaml
73	tests:
74	- agent: verifier
75	name: "VERDICT dondurmeli"
76	input: "Verify this build"
77	assertions:
78	- type: contains-any
79	values: ["VERDICT: PASS", "VERDICT: WARN", "VERDICT: FAIL"]
80
81	- agent: sleuth
82	name: "Root cause belirtmeli"
83	input: "Users can't login after deployment"
84	assertions:
85	- type: contains-any
86	values: ["root cause", "neden", "caused by"]
87	- type: contains
88	value: "file" # Dosya referansi olmali
89	```
90
91	### 4. Tutarlilik Testi
92
93	Ayni input'a farkli zamanlarda benzer cevap vermeli.
94
95	```yaml
96	# test-suites/consistency.yaml
97	tests:
98	- agent: architect
99	name: "Tutarli mimari tavsiye"
100	input: "Should I use microservices or monolith for a 3-person startup?"
101	runs: 3
102	assertions:
103	- type: consistent-sentiment
104	threshold: 0.8 # %80 tutarlilik
105	- type: contains-in-all
106	value: "monolith" # Her seferinde monolith onerilmeli (3 kisi icin)
107	```
108
109	## Test Calistirma
110
111	### Manuel Test
112
113	```bash
114	# Tek agent test
115	claude -p "$(cat agents/code-reviewer.md)
116
117	Test input: Review this code that has SQL injection" \
118	--no-input 2>/dev/null \| grep -c "injection"
119	# 1 veya daha fazla = PASS, 0 = FAIL
120	```
121
122	### Batch Test Script
123
124	```bash
125	#!/bin/bash
126	# scripts/agent-qa.sh
127
128	PASS=0
129	FAIL=0
130	TOTAL=0
131
132	run_test() {
133	local agent="$1"
134	local name="$2"
135	local input="$3"
136	local expected="$4"
137
138	TOTAL=$((TOTAL + 1))
139	local output=$(claude -p "$(cat agents/${agent}.md)
140
141	${input}" --no-input 2>/dev/null)
142
143	if echo "$output" \| grep -qi "$expected"; then
144	echo " PASS: $name"
145	PASS=$((PASS + 1))
146	else
147	echo " FAIL: $name (expected '$expected')"
148	FAIL=$((FAIL + 1))
149	fi
150	}
151
152	echo "=== Agent QA Test Suite ==="
153	echo ""
154
155	echo "[code-reviewer]"
156	run_test "code-reviewer" \
157	"SQL injection tespiti" \
158	"Review: db.query('SELECT * FROM users WHERE id=' + id)" \
159	"injection"
160
161	run_test "code-reviewer" \
162	"Severity belirtme" \
163	"Review: eval(req.body.code)" \
164	"CRITICAL\\|HIGH"
165
166	echo ""
167	echo "[verifier]"
168	run_test "verifier" \
169	"VERDICT dondurmeli" \
170	"Verify: all tests pass, build succeeds" \
171	"VERDICT"
172
173	echo ""
174	echo "Results: $PASS/$TOTAL passed, $FAIL failed"
175	```
176
177	## Regression Tespiti
178
179	### Baseline Olusturma
180
181	```bash
182	# Ilk calistirmada baseline kaydet
183	./scripts/agent-qa.sh > .claude/qa-baseline.txt
184
185	# Sonraki calistirmalarda karsilastir
186	./scripts/agent-qa.sh > /tmp/qa-current.txt
187	diff .claude/qa-baseline.txt /tmp/qa-current.txt
188	```
189
190	### Ne Zaman Test Et
191
192	\| Olay \| Test Skop \|
193	\|------\|-----------\|
194	\| Agent prompt degisti \| O agen

People who liked this also viewed

6 picks

Type

agent-qa-testing

519stars44forks

View on GitHub

How to install agent-qa-testing?

vibeeval/vibecosystem/agent-qa-testing

$npx -y skills add vibeeval/vibecosystem --skill agent-qa-testing

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Agent QA Testing
2
3	Agent'lar buyudukce "role drift" olur -- code-reviewer guvenlik yorumu yapar, architect kod yazar. Bu skill, agent'larin protokollerine uyumluluunu sistematik olarak test eder.
4
5	## Test Tipleri
6
7	### 1. Protokol Uyumluluk Testi
8
9	Agent'in system prompt'undaki kurallara uyup uymadigini test et.
10
11	```yaml
12	# test-suites/code-reviewer.yaml
13	agent: code-reviewer
14	tests:
15	- name: "Guvenlik bulgusunda severity belirtmeli"
16	input: "Review this code: app.get('/api/users/:id', (req, res) => { db.query('SELECT * FROM users WHERE id = ' + req.params.id) })"
17	assertions:
18	- type: contains
19	value: "SQL injection"
20	- type: contains-any
21	values: ["CRITICAL", "HIGH", "MEDIUM", "LOW"]
22	- type: not-contains
23	value: "looks good"
24
25	- name: "Kod yazmamali, sadece review etmeli"
26	input: "Review this function and rewrite it better"
27	assertions:
28	- type: not-contains
29	value: "```typescript" # Kod blogu olmamali
30	- type: contains-any
31	values: ["suggest", "recommend", "consider"] # Oneri vermeli
32	```
33
34	### 2. Rol Sinir Testi
35
36	Agent'in kendi rolunun disina cikip cikmadigini test et.
37
38	```yaml
39	# test-suites/role-boundaries.yaml
40	tests:
41	- agent: security-reviewer
42	name: "UI tasarim onerisi yapMAmali"
43	input: "This component looks ugly, should we change the colors?"
44	assertions:
45	- type: not-contains-any
46	values: ["color", "CSS", "style", "design"]
47	- type: contains-any
48	values: ["security", "out of scope", "not my domain"]
49
50	- agent: architect
51	name: "Direkt kod yazmamali, tasarim onerileri vermeli"
52	input: "Implement a caching layer for the API"
53	assertions:
54	- type: contains-any
55	values: ["pattern", "approach", "architecture", "design"]
56	- type: not-contains
57	value: "npm install"
58
59	- agent: tdd-guide
60	name: "Once test yazmali, sonra implementasyon"
61	input: "Add a login feature"
62	assertions:
63	- type: matches-order
64	values: ["test", "implement"] # test kelimesi implement'tan once gelmeli
65	```
66
67	### 3. Output Kalite Testi
68
69	Agent ciktisinin yapisal kalitesini test et.
70
71	```yaml
72	# test-suites/output-quality.yaml
73	tests:
74	- agent: verifier
75	name: "VERDICT dondurmeli"
76	input: "Verify this build"
77	assertions:
78	- type: contains-any
79	values: ["VERDICT: PASS", "VERDICT: WARN", "VERDICT: FAIL"]
80
81	- agent: sleuth
82	name: "Root cause belirtmeli"
83	input: "Users can't login after deployment"
84	assertions:
85	- type: contains-any
86	values: ["root cause", "neden", "caused by"]
87	- type: contains
88	value: "file" # Dosya referansi olmali
89	```
90
91	### 4. Tutarlilik Testi
92
93	Ayni input'a farkli zamanlarda benzer cevap vermeli.
94
95	```yaml
96	# test-suites/consistency.yaml
97	tests:
98	- agent: architect
99	name: "Tutarli mimari tavsiye"
100	input: "Should I use microservices or monolith for a 3-person startup?"
101	runs: 3
102	assertions:
103	- type: consistent-sentiment
104	threshold: 0.8 # %80 tutarlilik
105	- type: contains-in-all
106	value: "monolith" # Her seferinde monolith onerilmeli (3 kisi icin)
107	```
108
109	## Test Calistirma
110
111	### Manuel Test
112
113	```bash
114	# Tek agent test
115	claude -p "$(cat agents/code-reviewer.md)
116
117	Test input: Review this code that has SQL injection" \
118	--no-input 2>/dev/null \| grep -c "injection"
119	# 1 veya daha fazla = PASS, 0 = FAIL
120	```
121
122	### Batch Test Script
123
124	```bash
125	#!/bin/bash
126	# scripts/agent-qa.sh
127
128	PASS=0
129	FAIL=0
130	TOTAL=0
131
132	run_test() {
133	local agent="$1"
134	local name="$2"
135	local input="$3"
136	local expected="$4"
137
138	TOTAL=$((TOTAL + 1))
139	local output=$(claude -p "$(cat agents/${agent}.md)
140
141	${input}" --no-input 2>/dev/null)
142
143	if echo "$output" \| grep -qi "$expected"; then
144	echo " PASS: $name"
145	PASS=$((PASS + 1))
146	else
147	echo " FAIL: $name (expected '$expected')"
148	FAIL=$((FAIL + 1))
149	fi
150	}
151
152	echo "=== Agent QA Test Suite ==="
153	echo ""
154
155	echo "[code-reviewer]"
156	run_test "code-reviewer" \
157	"SQL injection tespiti" \
158	"Review: db.query('SELECT * FROM users WHERE id=' + id)" \
159	"injection"
160
161	run_test "code-reviewer" \
162	"Severity belirtme" \
163	"Review: eval(req.body.code)" \
164	"CRITICAL\\|HIGH"
165
166	echo ""
167	echo "[verifier]"
168	run_test "verifier" \
169	"VERDICT dondurmeli" \
170	"Verify: all tests pass, build succeeds" \
171	"VERDICT"
172
173	echo ""
174	echo "Results: $PASS/$TOTAL passed, $FAIL failed"
175	```
176
177	## Regression Tespiti
178
179	### Baseline Olusturma
180
181	```bash
182	# Ilk calistirmada baseline kaydet
183	./scripts/agent-qa.sh > .claude/qa-baseline.txt
184
185	# Sonraki calistirmalarda karsilastir
186	./scripts/agent-qa.sh > /tmp/qa-current.txt
187	diff .claude/qa-baseline.txt /tmp/qa-current.txt
188	```
189
190	### Ne Zaman Test Et
191
192	\| Olay \| Test Skop \|
193	\|------\|-----------\|
194	\| Agent prompt degisti \| O agen

People who liked this also viewed

6 picks

Type