hallucination-prevention

Name: hallucination-prevention
Author: DevelopersGlobal

64stars10forks

How to install hallucination-prevention?

developersglobal/ai-agent-skills/hallucination-prevention

$npx -y skills add DevelopersGlobal/ai-agent-skills --skill hallucination-prevention

Installs into the current project.

Summary

Detects and mitigates LLM hallucinations in production pipelines. Validates AI-generated facts, code, and decisions before they reach end users or downstream systems.

Files · 1

View on GitHub

SKILL.md

1	## Overview
2
3	LLMs hallucinate — they generate plausible-sounding but factually incorrect content with high confidence. In production systems, hallucinations cause incorrect medical advice, broken code, wrong legal citations, fabricated API calls, and erroneous financial data.
4
5	This skill builds hallucination detection and mitigation into your AI pipeline architecture — before output reaches users or downstream systems.
6
7	## When to Use
8
9	- Any AI pipeline where factual accuracy matters
10	- When agents generate code, SQL, or API calls that will be executed
11	- When agents retrieve information that will influence decisions
12	- RAG (Retrieval-Augmented Generation) systems
13	- Agentic pipelines with tool use and multi-step reasoning
14
15	## Process
16
17	### Step 1: Classify Your Hallucination Risk
18
19	1. Map all AI outputs in your system. For each, classify:
20	- Critical: Incorrect output causes harm (medical, legal, financial, code execution)
21	- High: Incorrect output wastes significant user/system resources
22	- Medium: Incorrect output is annoying but recoverable
23	- Low: Incorrect output is cosmetic
24	2. Apply this skill's full rigor to Critical and High outputs. Lighter checks for Medium and Low.
25
26	Verify: Every AI output in your pipeline has a hallucination risk classification.
27
28	### Step 2: Ground LLM Outputs
29
30	3. For factual queries: always provide source documents in the prompt (RAG pattern). Instruct the model to cite sources and refuse to answer if the information isn't in the provided documents.
31	4. For code generation: run the code in a sandbox and verify it produces the expected output before returning to the user.
32	5. For structured outputs (JSON, SQL, etc.): validate against a schema before using.
33	6. For numerical claims: require the model to show its work step-by-step and verify key calculations independently.
34
35	Verify: Every Critical/High output is grounded in provided sources or verified by execution.
36
37	### Step 3: Build Verification Layers
38
39	7. Self-consistency checking: Ask the model the same question 2–3 ways and compare answers. Divergent answers signal uncertain ground.
40	8. Chain-of-thought with verification: Prompt the model to reason step-by-step, then verify the reasoning chain, not just the conclusion.
41	9. Separate generation from verification: Use one prompt to generate, a different prompt to critically evaluate. Never use the same model call for both.
42	10. Human-in-the-loop for Critical outputs: Require human review before Critical AI-generated content reaches end users.
43
44	Verify: Critical outputs have at least two independent verification mechanisms.
45
46	### Step 4: Design Fail-Safe Defaults
47
48	11. When the model is uncertain, it should say so — never present uncertain information as confident fact.
49	- Prompt pattern: "If you're not sure, say 'I don't know' rather than guessing."
50	12. Confidence scores: Where available, expose confidence thresholds. Route low-confidence outputs to human review.
51	13. Graceful degradation: If the AI cannot answer accurately, show the user relevant source documents instead.
52
53	Verify: The system has a defined behavior for when AI confidence is low.
54
55	### Step 5: Monitor for Hallucinations in Production
56
57	14. Log all AI inputs and outputs.
58	15. Implement feedback mechanisms (thumbs up/down, explicit corrections).
59	16. Regularly audit a sample of outputs for accuracy.
60	17. Track and alert on: sudden changes in output patterns, high user correction rates, model responses contradicting grounded sources.
61
62	Verify: A feedback mechanism exists and hallucination incidents are tracked.
63
64	## Common Rationalizations (and Rebuttals)
65
66	\| Excuse \| Rebuttal \|
67	\|--------\|----------\|
68	\| "This model is highly accurate" \| All LLMs hallucinate. Accuracy ≠ 100%. Design for the failure case. \|
69	\| "Users will catch errors" \| Users often can't distinguish hallucinated content from real content. Don't rely on them. \|
70	\| "It's a demo/MVP" \| Hallucinations that reach demos become the baseline expectation. Fix them early. \|
71	\| "We'll add verification in v2" \| Hallucinations that cause harm in v1 may not give you a v2. \|
72
73	## Red Flags
74
75	- AI-generated code is executed without testing or sandbox verification
76	- AI-generated facts are displayed without source citations
77	- No human review for medical, legal, or financial AI outputs
78	- The same model is used to generate and verify its own output
79	- No logging or monitoring of AI outputs in production
80
81	## Verification
82
83	- [ ] All AI outputs classified by hallucination risk
84	- [ ] Critical/High outputs are grounded in source documents or execution
85	- [ ] At least two independent verification mechanisms for Critical outputs
86	- [ ] System h

People who liked this also viewed

6 picks

Type

hallucination-prevention

64stars10forks

View on GitHub

How to install hallucination-prevention?

developersglobal/ai-agent-skills/hallucination-prevention

$npx -y skills add DevelopersGlobal/ai-agent-skills --skill hallucination-prevention

Installs into the current project.

Summary

Detects and mitigates LLM hallucinations in production pipelines. Validates AI-generated facts, code, and decisions before they reach end users or downstream systems.

Files · 1

View on GitHub

SKILL.md

1	## Overview
2
3	LLMs hallucinate — they generate plausible-sounding but factually incorrect content with high confidence. In production systems, hallucinations cause incorrect medical advice, broken code, wrong legal citations, fabricated API calls, and erroneous financial data.
4
5	This skill builds hallucination detection and mitigation into your AI pipeline architecture — before output reaches users or downstream systems.
6
7	## When to Use
8
9	- Any AI pipeline where factual accuracy matters
10	- When agents generate code, SQL, or API calls that will be executed
11	- When agents retrieve information that will influence decisions
12	- RAG (Retrieval-Augmented Generation) systems
13	- Agentic pipelines with tool use and multi-step reasoning
14
15	## Process
16
17	### Step 1: Classify Your Hallucination Risk
18
19	1. Map all AI outputs in your system. For each, classify:
20	- Critical: Incorrect output causes harm (medical, legal, financial, code execution)
21	- High: Incorrect output wastes significant user/system resources
22	- Medium: Incorrect output is annoying but recoverable
23	- Low: Incorrect output is cosmetic
24	2. Apply this skill's full rigor to Critical and High outputs. Lighter checks for Medium and Low.
25
26	Verify: Every AI output in your pipeline has a hallucination risk classification.
27
28	### Step 2: Ground LLM Outputs
29
30	3. For factual queries: always provide source documents in the prompt (RAG pattern). Instruct the model to cite sources and refuse to answer if the information isn't in the provided documents.
31	4. For code generation: run the code in a sandbox and verify it produces the expected output before returning to the user.
32	5. For structured outputs (JSON, SQL, etc.): validate against a schema before using.
33	6. For numerical claims: require the model to show its work step-by-step and verify key calculations independently.
34
35	Verify: Every Critical/High output is grounded in provided sources or verified by execution.
36
37	### Step 3: Build Verification Layers
38
39	7. Self-consistency checking: Ask the model the same question 2–3 ways and compare answers. Divergent answers signal uncertain ground.
40	8. Chain-of-thought with verification: Prompt the model to reason step-by-step, then verify the reasoning chain, not just the conclusion.
41	9. Separate generation from verification: Use one prompt to generate, a different prompt to critically evaluate. Never use the same model call for both.
42	10. Human-in-the-loop for Critical outputs: Require human review before Critical AI-generated content reaches end users.
43
44	Verify: Critical outputs have at least two independent verification mechanisms.
45
46	### Step 4: Design Fail-Safe Defaults
47
48	11. When the model is uncertain, it should say so — never present uncertain information as confident fact.
49	- Prompt pattern: "If you're not sure, say 'I don't know' rather than guessing."
50	12. Confidence scores: Where available, expose confidence thresholds. Route low-confidence outputs to human review.
51	13. Graceful degradation: If the AI cannot answer accurately, show the user relevant source documents instead.
52
53	Verify: The system has a defined behavior for when AI confidence is low.
54
55	### Step 5: Monitor for Hallucinations in Production
56
57	14. Log all AI inputs and outputs.
58	15. Implement feedback mechanisms (thumbs up/down, explicit corrections).
59	16. Regularly audit a sample of outputs for accuracy.
60	17. Track and alert on: sudden changes in output patterns, high user correction rates, model responses contradicting grounded sources.
61
62	Verify: A feedback mechanism exists and hallucination incidents are tracked.
63
64	## Common Rationalizations (and Rebuttals)
65
66	\| Excuse \| Rebuttal \|
67	\|--------\|----------\|
68	\| "This model is highly accurate" \| All LLMs hallucinate. Accuracy ≠ 100%. Design for the failure case. \|
69	\| "Users will catch errors" \| Users often can't distinguish hallucinated content from real content. Don't rely on them. \|
70	\| "It's a demo/MVP" \| Hallucinations that reach demos become the baseline expectation. Fix them early. \|
71	\| "We'll add verification in v2" \| Hallucinations that cause harm in v1 may not give you a v2. \|
72
73	## Red Flags
74
75	- AI-generated code is executed without testing or sandbox verification
76	- AI-generated facts are displayed without source citations
77	- No human review for medical, legal, or financial AI outputs
78	- The same model is used to generate and verify its own output
79	- No logging or monitoring of AI outputs in production
80
81	## Verification
82
83	- [ ] All AI outputs classified by hallucination risk
84	- [ ] Critical/High outputs are grounded in source documents or execution
85	- [ ] At least two independent verification mechanisms for Critical outputs
86	- [ ] System h

People who liked this also viewed

6 picks

Type