rag-architect

Name: rag-architect
Author: Jeffallan

11kstars971forks

How to install rag-architect?

jeffallan/claude-skills/rag-architect

$npx -y skills add Jeffallan/claude-skills --skill rag-architect

Installs into the current project.

Summary

Designs and implements production-grade RAG systems by chunking documents, generating embeddings, configuring vector stores, building hybrid search pipelines, applying reranking, and evaluating retrieval quality. Use when building RAG systems, vector databases, or knowledge-groun

Files · 1

View on GitHub

SKILL.md

1	# RAG Architect
2
3	## Core Workflow
4
5	1. Requirements Analysis — Identify retrieval needs, latency constraints, accuracy requirements, and scale
6	2. Vector Store Design — Select database, schema design, indexing strategy, sharding approach
7	3. Chunking Strategy — Document splitting, overlap, semantic boundaries, metadata enrichment
8	4. Retrieval Pipeline — Embedding selection, query transformation, hybrid search, reranking
9	5. Evaluation & Iteration — Metrics tracking, retrieval debugging, continuous optimization
10
11	For each step, validate before moving on (see checkpoints below).
12
13	## Reference Guide
14
15	Load detailed guidance based on context:
16
17	\| Topic \| Reference \| Load When \|
18	\|-------\|-----------\|-----------\|
19	\| Vector Databases \| `references/vector-databases.md` \| Comparing Pinecone, Weaviate, Chroma, pgvector, Qdrant \|
20	\| Embedding Models \| `references/embedding-models.md` \| Selecting embeddings, fine-tuning, dimension trade-offs \|
21	\| Chunking Strategies \| `references/chunking-strategies.md` \| Document splitting, overlap, semantic chunking \|
22	\| Retrieval Optimization \| `references/retrieval-optimization.md` \| Hybrid search, reranking, query expansion, filtering \|
23	\| RAG Evaluation \| `references/rag-evaluation.md` \| Metrics, evaluation frameworks, debugging retrieval \|
24
25	## Implementation Examples
26
27	### 1. Chunking Documents
28
29	```python
30	from langchain.text_splitter import RecursiveCharacterTextSplitter
31
32	# Evaluate chunk_size on your domain data — never use 512 blindly
33	splitter = RecursiveCharacterTextSplitter(
34	chunk_size=800,
35	chunk_overlap=100,
36	separators=["\n\n", "\n", ". ", " "],
37	)
38
39	chunks = splitter.create_documents(
40	texts=[doc.page_content for doc in raw_docs],
41	metadatas=[{"source": doc.metadata["source"], "timestamp": doc.metadata.get("timestamp")} for doc in raw_docs],
42	)
43	```
44
45	Checkpoint: `assert all(c.metadata.get("source") for c in chunks), "Missing source metadata"`
46
47	### 2. Generating Embeddings & Indexing
48
49	```python
50	from openai import OpenAI
51	import qdrant_client
52	from qdrant_client.models import VectorParams, Distance, PointStruct
53
54	client = OpenAI()
55	qdrant = qdrant_client.QdrantClient("localhost", port=6333)
56
57	# Create collection
58	qdrant.recreate_collection(
59	collection_name="knowledge_base",
60	vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
61	)
62
63	def embed_chunks(chunks: list[str], model: str = "text-embedding-3-small") -> list[list[float]]:
64	response = client.embeddings.create(input=chunks, model=model)
65	return [r.embedding for r in response.data]
66
67	# Idempotent upsert with deduplication via deterministic IDs
68	import hashlib, uuid
69
70	points = []
71	for i, chunk in enumerate(chunks):
72	doc_id = str(uuid.UUID(hashlib.md5(chunk.page_content.encode()).hexdigest()))
73	embedding = embed_chunks([chunk.page_content])[0]
74	points.append(PointStruct(id=doc_id, vector=embedding, payload=chunk.metadata))
75
76	qdrant.upsert(collection_name="knowledge_base", points=points)
77	```
78
79	Checkpoint: `assert qdrant.count("knowledge_base").count == len(set(p.id for p in points)), "Deduplication failed"`
80
81	### 3. Hybrid Search (Vector + BM25)
82
83	```python
84	from qdrant_client.models import Filter, FieldCondition, MatchValue, SparseVector
85	from rank_bm25 import BM25Okapi
86
87	def hybrid_search(query: str, tenant_id: str, top_k: int = 20) -> list:
88	# Dense retrieval
89	query_embedding = embed_chunks([query])[0]
90	tenant_filter = Filter(must=[FieldCondition(key="tenant_id", match=MatchValue(value=tenant_id))])
91	dense_results = qdrant.search(
92	collection_name="knowledge_base",
93	query_vector=query_embedding,
94	query_filter=tenant_filter,
95	limit=top_k,
96	)
97
98	# Sparse retrieval (BM25)
99	corpus = [r.payload.get("text", "") for r in dense_results]
100	bm25 = BM25Okapi([doc.split() for doc in corpus])
101	bm25_scores = bm25.get_scores(query.split())
102
103	# Reciprocal Rank Fusion
104	ranked = sorted(
105	zip(dense_results, bm25_scores),
106	key=lambda x: 0.6 * x[0].score + 0.4 * x[1],
107	reverse=True,
108	)
109	return [r for r, _ in ranked[:top_k]]
110	```
111
112	**Checkpo

People who liked this also viewed

6 picks

Type

rag-architect

11kstars971forks

View on GitHub

How to install rag-architect?

jeffallan/claude-skills/rag-architect

$npx -y skills add Jeffallan/claude-skills --skill rag-architect

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# RAG Architect
2
3	## Core Workflow
4
5	1. Requirements Analysis — Identify retrieval needs, latency constraints, accuracy requirements, and scale
6	2. Vector Store Design — Select database, schema design, indexing strategy, sharding approach
7	3. Chunking Strategy — Document splitting, overlap, semantic boundaries, metadata enrichment
8	4. Retrieval Pipeline — Embedding selection, query transformation, hybrid search, reranking
9	5. Evaluation & Iteration — Metrics tracking, retrieval debugging, continuous optimization
10
11	For each step, validate before moving on (see checkpoints below).
12
13	## Reference Guide
14
15	Load detailed guidance based on context:
16
17	\| Topic \| Reference \| Load When \|
18	\|-------\|-----------\|-----------\|
19	\| Vector Databases \| `references/vector-databases.md` \| Comparing Pinecone, Weaviate, Chroma, pgvector, Qdrant \|
20	\| Embedding Models \| `references/embedding-models.md` \| Selecting embeddings, fine-tuning, dimension trade-offs \|
21	\| Chunking Strategies \| `references/chunking-strategies.md` \| Document splitting, overlap, semantic chunking \|
22	\| Retrieval Optimization \| `references/retrieval-optimization.md` \| Hybrid search, reranking, query expansion, filtering \|
23	\| RAG Evaluation \| `references/rag-evaluation.md` \| Metrics, evaluation frameworks, debugging retrieval \|
24
25	## Implementation Examples
26
27	### 1. Chunking Documents
28
29	```python
30	from langchain.text_splitter import RecursiveCharacterTextSplitter
31
32	# Evaluate chunk_size on your domain data — never use 512 blindly
33	splitter = RecursiveCharacterTextSplitter(
34	chunk_size=800,
35	chunk_overlap=100,
36	separators=["\n\n", "\n", ". ", " "],
37	)
38
39	chunks = splitter.create_documents(
40	texts=[doc.page_content for doc in raw_docs],
41	metadatas=[{"source": doc.metadata["source"], "timestamp": doc.metadata.get("timestamp")} for doc in raw_docs],
42	)
43	```
44
45	Checkpoint: `assert all(c.metadata.get("source") for c in chunks), "Missing source metadata"`
46
47	### 2. Generating Embeddings & Indexing
48
49	```python
50	from openai import OpenAI
51	import qdrant_client
52	from qdrant_client.models import VectorParams, Distance, PointStruct
53
54	client = OpenAI()
55	qdrant = qdrant_client.QdrantClient("localhost", port=6333)
56
57	# Create collection
58	qdrant.recreate_collection(
59	collection_name="knowledge_base",
60	vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
61	)
62
63	def embed_chunks(chunks: list[str], model: str = "text-embedding-3-small") -> list[list[float]]:
64	response = client.embeddings.create(input=chunks, model=model)
65	return [r.embedding for r in response.data]
66
67	# Idempotent upsert with deduplication via deterministic IDs
68	import hashlib, uuid
69
70	points = []
71	for i, chunk in enumerate(chunks):
72	doc_id = str(uuid.UUID(hashlib.md5(chunk.page_content.encode()).hexdigest()))
73	embedding = embed_chunks([chunk.page_content])[0]
74	points.append(PointStruct(id=doc_id, vector=embedding, payload=chunk.metadata))
75
76	qdrant.upsert(collection_name="knowledge_base", points=points)
77	```
78
79	Checkpoint: `assert qdrant.count("knowledge_base").count == len(set(p.id for p in points)), "Deduplication failed"`
80
81	### 3. Hybrid Search (Vector + BM25)
82
83	```python
84	from qdrant_client.models import Filter, FieldCondition, MatchValue, SparseVector
85	from rank_bm25 import BM25Okapi
86
87	def hybrid_search(query: str, tenant_id: str, top_k: int = 20) -> list:
88	# Dense retrieval
89	query_embedding = embed_chunks([query])[0]
90	tenant_filter = Filter(must=[FieldCondition(key="tenant_id", match=MatchValue(value=tenant_id))])
91	dense_results = qdrant.search(
92	collection_name="knowledge_base",
93	query_vector=query_embedding,
94	query_filter=tenant_filter,
95	limit=top_k,
96	)
97
98	# Sparse retrieval (BM25)
99	corpus = [r.payload.get("text", "") for r in dense_results]
100	bm25 = BM25Okapi([doc.split() for doc in corpus])
101	bm25_scores = bm25.get_scores(query.split())
102
103	# Reciprocal Rank Fusion
104	ranked = sorted(
105	zip(dense_results, bm25_scores),
106	key=lambda x: 0.6 * x[0].score + 0.4 * x[1],
107	reverse=True,
108	)
109	return [r for r, _ in ranked[:top_k]]
110	```
111
112	**Checkpo

People who liked this also viewed

6 picks

Type