ai-threat-testing

Name: ai-threat-testing
Author: transilienceai

423stars74forks

How to install ai-threat-testing?

transilienceai/communitytools/ai-threat-testing

$npx -y skills add transilienceai/communitytools --skill ai-threat-testing

Installs into the current project.

Summary

Offensive AI security testing and exploitation framework. Systematically tests LLM applications for OWASP Top 10 vulnerabilities including prompt injection, model extraction, data poisoning, and supply chain attacks. Integrates with pentest workflows to discover and exploit AI-sp

Files · 1

View on GitHub

SKILL.md

1	# AI Threat Testing
2
3	Test LLM applications for OWASP LLM Top 10 vulnerabilities using 10 specialized agents. Use for authorized AI security assessments.
4
5	## Quick Start
6
7	```
8	1. Specify target (LLM app URL, API endpoint, or local model)
9	2. Select scope: Full OWASP Top 10 \| Specific vulnerability \| Supply chain
10	3. Agents deploy, test, capture evidence
11	4. Professional report with PoCs generated
12	```
13
14	## Primary Agents
15
16	Each agent targets one OWASP LLM vulnerability:
17
18	1. Prompt Injection (LLM01): Direct/indirect injection, system prompt extraction
19	2. Output Handling (LLM02): Code/XSS injection, unsafe deserialization
20	3. Training Poisoning (LLM03): Membership inference, backdoors, data extraction
21	4. Resource Exhaustion (LLM04): Token flooding, DoS, cost impact
22	5. Supply Chain (LLM05): Dependency scanning, plugin security
23	6. Excessive Agency (LLM06): Privilege escalation, unauthorized actions
24	7. Model Extraction (LLM07): Query-based theft, data reconstruction
25	8. Vector Poisoning (LLM08): RAG injection, retrieval manipulation
26	9. Overreliance (LLM09): Hallucination testing, confidence manipulation
27	10. Logging Bypass (LLM10): Monitoring evasion, forensic gaps
28
29	See `reference/llm0X-*.md` for attack playbooks.
30
31	## Workflows
32
33	Full Assessment (4-8 hours):
34	```
35	- [ ] Reconnaissance
36	- [ ] Deploy all 10 agents
37	- [ ] Execute exploits
38	- [ ] Capture evidence
39	- [ ] Generate report
40	```
41
42	Focused Testing (1-3 hours):
43	```
44	- [ ] Select vulnerability (LLM01-10)
45	- [ ] Deploy agent
46	- [ ] Execute techniques
47	- [ ] Document findings
48	```
49
50	Supply Chain Audit (2-4 hours):
51	```
52	- [ ] Inventory dependencies
53	- [ ] Scan CVEs
54	- [ ] Test plugins/APIs
55	- [ ] Verify model provenance
56	```
57
58	## Integration
59
60	Enhances `/pentest` with AI-specific testing:
61	- Traditional pentesting + AI threat testing = complete security assessment
62	- Chain vulnerabilities across traditional and AI vectors
63	- Unified reporting with CVSS scores
64
65	## Key Techniques
66
67	Prompt Injection: Instruction override, system prompt extraction, filter evasion
68	Model Extraction: Query sampling, token analysis, membership inference
69	Data Poisoning: Behavioral anomalies, backdoor triggers, bias analysis
70	DoS: Token flooding, recursive expansion, context exhaustion
71	Supply Chain: CVE scanning, plugin audit, model verification
72	MCP Tool Abuse: MCP server inspectors/debuggers often expose `/api/mcp/connect` or similar endpoints that accept `serverConfig` with arbitrary `command` parameters — unauthenticated RCE. Check for MCP Inspector, MCP Playground, or any MCP debugging UI on non-standard ports (6274, 3000, etc.).
73
74	## Evidence Capture
75
76	All agents collect: screenshots, network logs, API responses, errors, console output, execution metrics.
77
78	## Reporting
79
80	Automated reports include: executive summary, detailed findings (CVSS scores), PoC scripts, evidence, remediation guidance.
81
82	## Critical Rules
83
84	- Written authorization REQUIRED before testing
85	- Never exceed defined scope
86	- Test in isolated environments when possible
87	- Document all findings with reproducible PoCs
88	- Follow responsible disclosure practices
89
90	## Integration
91
92	- Integrates with `/pentest` skill for comprehensive security testing
93	- AI-specific vulnerability knowledge in `/AGENTS.md`
94	- Attack playbooks in `reference/llm0X-*.md`

People who liked this also viewed

6 picks

Type

ai-threat-testing

423stars74forks

View on GitHub

How to install ai-threat-testing?

transilienceai/communitytools/ai-threat-testing

$npx -y skills add transilienceai/communitytools --skill ai-threat-testing

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# AI Threat Testing
2
3	Test LLM applications for OWASP LLM Top 10 vulnerabilities using 10 specialized agents. Use for authorized AI security assessments.
4
5	## Quick Start
6
7	```
8	1. Specify target (LLM app URL, API endpoint, or local model)
9	2. Select scope: Full OWASP Top 10 \| Specific vulnerability \| Supply chain
10	3. Agents deploy, test, capture evidence
11	4. Professional report with PoCs generated
12	```
13
14	## Primary Agents
15
16	Each agent targets one OWASP LLM vulnerability:
17
18	1. Prompt Injection (LLM01): Direct/indirect injection, system prompt extraction
19	2. Output Handling (LLM02): Code/XSS injection, unsafe deserialization
20	3. Training Poisoning (LLM03): Membership inference, backdoors, data extraction
21	4. Resource Exhaustion (LLM04): Token flooding, DoS, cost impact
22	5. Supply Chain (LLM05): Dependency scanning, plugin security
23	6. Excessive Agency (LLM06): Privilege escalation, unauthorized actions
24	7. Model Extraction (LLM07): Query-based theft, data reconstruction
25	8. Vector Poisoning (LLM08): RAG injection, retrieval manipulation
26	9. Overreliance (LLM09): Hallucination testing, confidence manipulation
27	10. Logging Bypass (LLM10): Monitoring evasion, forensic gaps
28
29	See `reference/llm0X-*.md` for attack playbooks.
30
31	## Workflows
32
33	Full Assessment (4-8 hours):
34	```
35	- [ ] Reconnaissance
36	- [ ] Deploy all 10 agents
37	- [ ] Execute exploits
38	- [ ] Capture evidence
39	- [ ] Generate report
40	```
41
42	Focused Testing (1-3 hours):
43	```
44	- [ ] Select vulnerability (LLM01-10)
45	- [ ] Deploy agent
46	- [ ] Execute techniques
47	- [ ] Document findings
48	```
49
50	Supply Chain Audit (2-4 hours):
51	```
52	- [ ] Inventory dependencies
53	- [ ] Scan CVEs
54	- [ ] Test plugins/APIs
55	- [ ] Verify model provenance
56	```
57
58	## Integration
59
60	Enhances `/pentest` with AI-specific testing:
61	- Traditional pentesting + AI threat testing = complete security assessment
62	- Chain vulnerabilities across traditional and AI vectors
63	- Unified reporting with CVSS scores
64
65	## Key Techniques
66
67	Prompt Injection: Instruction override, system prompt extraction, filter evasion
68	Model Extraction: Query sampling, token analysis, membership inference
69	Data Poisoning: Behavioral anomalies, backdoor triggers, bias analysis
70	DoS: Token flooding, recursive expansion, context exhaustion
71	Supply Chain: CVE scanning, plugin audit, model verification
72	MCP Tool Abuse: MCP server inspectors/debuggers often expose `/api/mcp/connect` or similar endpoints that accept `serverConfig` with arbitrary `command` parameters — unauthenticated RCE. Check for MCP Inspector, MCP Playground, or any MCP debugging UI on non-standard ports (6274, 3000, etc.).
73
74	## Evidence Capture
75
76	All agents collect: screenshots, network logs, API responses, errors, console output, execution metrics.
77
78	## Reporting
79
80	Automated reports include: executive summary, detailed findings (CVSS scores), PoC scripts, evidence, remediation guidance.
81
82	## Critical Rules
83
84	- Written authorization REQUIRED before testing
85	- Never exceed defined scope
86	- Test in isolated environments when possible
87	- Document all findings with reproducible PoCs
88	- Follow responsible disclosure practices
89
90	## Integration
91
92	- Integrates with `/pentest` skill for comprehensive security testing
93	- AI-specific vulnerability knowledge in `/AGENTS.md`
94	- Attack playbooks in `reference/llm0X-*.md`

People who liked this also viewed

6 picks

Type