16-ai-llm-security

Name: 16-ai-llm-security
Author: Masriyan

190stars37forks

How to install 16-ai-llm-security?

masriyan/claude-code-cybersecurity-skill/16-ai-llm-security

$npx -y skills add Masriyan/Claude-Code-CyberSecurity-Skill --skill 16-ai-llm-security

Installs into the current project.

Summary

LLM and AI application security testing — prompt injection, jailbreak resistance, OWASP LLM Top 10 (2025), RAG and agent/tool-use security, model supply chain, and AI red teaming for authorized assessments

Files · 1

View on GitHub

SKILL.md

1	# AI & LLM Security
2
3	## Purpose
4
5	Enable Claude to assess the security of AI/LLM-powered applications — chatbots, RAG pipelines, autonomous agents, and tool-using systems. Claude maps findings to the OWASP Top 10 for LLM Applications (2025) and the MITRE ATLAS adversarial-ML knowledge base, builds reproducible attack cases, and recommends concrete mitigations (input/output guardrails, least-privilege tool scopes, content provenance).
6
7	> Authorization Required: Only test AI systems you own or are explicitly authorized to assess. Prompt-injection and data-exfiltration testing against third-party AI services may violate their terms of service and local law. Confirm written scope before proceeding.
8
9	---
10
11	## Activation Triggers
12
13	This skill activates when the user asks about:
14	- Prompt injection (direct or indirect), jailbreaks, or system-prompt extraction
15	- OWASP LLM Top 10, MITRE ATLAS, or AI/ML threat modeling
16	- Securing a RAG pipeline, vector database, or retrieval layer
17	- LLM agent / tool-use / function-calling security and confused-deputy risks
18	- Guardrail, content-filter, or model output validation design
19	- Sensitive-information disclosure or training-data leakage from a model
20	- Model / ML supply chain security (model files, `pickle`, model registries)
21	- AI red teaming, jailbreak corpora, or automated adversarial prompt generation
22	- Securing MCP (Model Context Protocol) servers and tool integrations
23
24	---
25
26	## Prerequisites
27
28	```bash
29	pip install requests pyyaml rich
30	```
31
32	Optional enhanced capabilities:
33	- `garak` — LLM vulnerability scanner (NVIDIA)
34	- `promptfoo` — prompt/red-team evaluation harness
35	- API key for the target LLM endpoint (test environment only)
36	- `modelscan` / `picklescan` — ML model file safety scanning
37
38	---
39
40	## Core Capabilities
41
42	### 1. Threat Modeling (OWASP LLM Top 10 — 2025)
43
44	When asked to threat-model an AI application, map the system against each category and record exposure:
45
46	\| ID \| Risk \| What to look for \|
47	\|----\|------\|------------------\|
48	\| LLM01 \| Prompt Injection \| Untrusted text reaching the prompt (direct & indirect via RAG/web/email) \|
49	\| LLM02 \| Sensitive Information Disclosure \| PII/secrets in prompts, outputs, or training data; system-prompt leakage \|
50	\| LLM03 \| Supply Chain \| Untrusted models, LoRA adapters, datasets, plugins, `pickle` deserialization \|
51	\| LLM04 \| Data & Model Poisoning \| Tainted training/fine-tune/RAG data; backdoors \|
52	\| LLM05 \| Improper Output Handling \| LLM output passed unsanitized to SQL, shell, browser (XSS), or `eval` \|
53	\| LLM06 \| Excessive Agency \| Over-broad tool scopes, autonomous side effects, no human-in-the-loop \|
54	\| LLM07 \| System Prompt Leakage \| Secrets/authz logic embedded in the system prompt \|
55	\| LLM08 \| Vector & Embedding Weaknesses \| RAG access-control bypass, embedding inversion, cross-tenant leakage \|
56	\| LLM09 \| Misinformation \| Hallucinations relied on for security/safety decisions \|
57	\| LLM10 \| Unbounded Consumption \| Cost/DoS via token floods, model extraction, wallet-drain \|
58
59	Produce a per-category table: Exposure (Yes/No/Partial) → Evidence → Severity → Mitigation.
60
61	### 2. Prompt Injection & Jailbreak Testing
62
63	Direct injection — user input that overrides instructions. Test families:
64	- Instruction override ("ignore previous instructions and …")
65	- Role-play / persona escape (DAN-style, hypothetical framing)
66	- Encoding/obfuscation (Base64, ROT13, leetspeak, homoglyphs, zero-width chars)
67	- Token smuggling and prompt-boundary confusion (fake delimiters, fake system tags)
68	- Many-shot jailbreaking (long context of faux dialogue priming compliance)
69	- Crescendo / multi-turn gradual escalation
70
71	Indirect injection — payload arrives via retrieved/processed content (web page, PDF, email, RAG doc, tool output). This is the highest-impact class for agents. Test that retrieved text cannot issue commands, exfiltrate context, or trigger tools.
72
73	For every test record: payload, channel (direct/indirect), goal (override / exfiltrate / tool-abuse), and result (blocked / partial / success). Use `scripts/prompt_injection_tester.py` to run a corpus and score outcomes.
74
75	Refusal-quality note: a single refusal is not a pass. Re-test the same goal across ≥3 phrasings and obfuscations before marking a control effective.
76
77	### 3. RAG & Vector Store Security
78
79	When reviewing a RAG pipeline:
80	1. Access control at retrieval — confirm the vector query is filtered by the caller's permissions, not just the app's. Test cross-tenant / cross-user document leakage.
81	2. Indirect injection surface — treat every ingested document as attacker-controlled; v

People who liked this also viewed

6 picks

Type

16-ai-llm-security

190stars37forks

View on GitHub

How to install 16-ai-llm-security?

masriyan/claude-code-cybersecurity-skill/16-ai-llm-security

$npx -y skills add Masriyan/Claude-Code-CyberSecurity-Skill --skill 16-ai-llm-security

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# AI & LLM Security
2
3	## Purpose
4
5	Enable Claude to assess the security of AI/LLM-powered applications — chatbots, RAG pipelines, autonomous agents, and tool-using systems. Claude maps findings to the OWASP Top 10 for LLM Applications (2025) and the MITRE ATLAS adversarial-ML knowledge base, builds reproducible attack cases, and recommends concrete mitigations (input/output guardrails, least-privilege tool scopes, content provenance).
6
7	> Authorization Required: Only test AI systems you own or are explicitly authorized to assess. Prompt-injection and data-exfiltration testing against third-party AI services may violate their terms of service and local law. Confirm written scope before proceeding.
8
9	---
10
11	## Activation Triggers
12
13	This skill activates when the user asks about:
14	- Prompt injection (direct or indirect), jailbreaks, or system-prompt extraction
15	- OWASP LLM Top 10, MITRE ATLAS, or AI/ML threat modeling
16	- Securing a RAG pipeline, vector database, or retrieval layer
17	- LLM agent / tool-use / function-calling security and confused-deputy risks
18	- Guardrail, content-filter, or model output validation design
19	- Sensitive-information disclosure or training-data leakage from a model
20	- Model / ML supply chain security (model files, `pickle`, model registries)
21	- AI red teaming, jailbreak corpora, or automated adversarial prompt generation
22	- Securing MCP (Model Context Protocol) servers and tool integrations
23
24	---
25
26	## Prerequisites
27
28	```bash
29	pip install requests pyyaml rich
30	```
31
32	Optional enhanced capabilities:
33	- `garak` — LLM vulnerability scanner (NVIDIA)
34	- `promptfoo` — prompt/red-team evaluation harness
35	- API key for the target LLM endpoint (test environment only)
36	- `modelscan` / `picklescan` — ML model file safety scanning
37
38	---
39
40	## Core Capabilities
41
42	### 1. Threat Modeling (OWASP LLM Top 10 — 2025)
43
44	When asked to threat-model an AI application, map the system against each category and record exposure:
45
46	\| ID \| Risk \| What to look for \|
47	\|----\|------\|------------------\|
48	\| LLM01 \| Prompt Injection \| Untrusted text reaching the prompt (direct & indirect via RAG/web/email) \|
49	\| LLM02 \| Sensitive Information Disclosure \| PII/secrets in prompts, outputs, or training data; system-prompt leakage \|
50	\| LLM03 \| Supply Chain \| Untrusted models, LoRA adapters, datasets, plugins, `pickle` deserialization \|
51	\| LLM04 \| Data & Model Poisoning \| Tainted training/fine-tune/RAG data; backdoors \|
52	\| LLM05 \| Improper Output Handling \| LLM output passed unsanitized to SQL, shell, browser (XSS), or `eval` \|
53	\| LLM06 \| Excessive Agency \| Over-broad tool scopes, autonomous side effects, no human-in-the-loop \|
54	\| LLM07 \| System Prompt Leakage \| Secrets/authz logic embedded in the system prompt \|
55	\| LLM08 \| Vector & Embedding Weaknesses \| RAG access-control bypass, embedding inversion, cross-tenant leakage \|
56	\| LLM09 \| Misinformation \| Hallucinations relied on for security/safety decisions \|
57	\| LLM10 \| Unbounded Consumption \| Cost/DoS via token floods, model extraction, wallet-drain \|
58
59	Produce a per-category table: Exposure (Yes/No/Partial) → Evidence → Severity → Mitigation.
60
61	### 2. Prompt Injection & Jailbreak Testing
62
63	Direct injection — user input that overrides instructions. Test families:
64	- Instruction override ("ignore previous instructions and …")
65	- Role-play / persona escape (DAN-style, hypothetical framing)
66	- Encoding/obfuscation (Base64, ROT13, leetspeak, homoglyphs, zero-width chars)
67	- Token smuggling and prompt-boundary confusion (fake delimiters, fake system tags)
68	- Many-shot jailbreaking (long context of faux dialogue priming compliance)
69	- Crescendo / multi-turn gradual escalation
70
71	Indirect injection — payload arrives via retrieved/processed content (web page, PDF, email, RAG doc, tool output). This is the highest-impact class for agents. Test that retrieved text cannot issue commands, exfiltrate context, or trigger tools.
72
73	For every test record: payload, channel (direct/indirect), goal (override / exfiltrate / tool-abuse), and result (blocked / partial / success). Use `scripts/prompt_injection_tester.py` to run a corpus and score outcomes.
74
75	Refusal-quality note: a single refusal is not a pass. Re-test the same goal across ≥3 phrasings and obfuscations before marking a control effective.
76
77	### 3. RAG & Vector Store Security
78
79	When reviewing a RAG pipeline:
80	1. Access control at retrieval — confirm the vector query is filtered by the caller's permissions, not just the app's. Test cross-tenant / cross-user document leakage.
81	2. Indirect injection surface — treat every ingested document as attacker-controlled; v

People who liked this also viewed

6 picks

Type