agentdb-optimization

Name: agentdb-optimization
Author: spencermarx

301stars24forks

How to install agentdb-optimization?

spencermarx/open-code-review/agentdb-optimization

$npx -y skills add spencermarx/open-code-review --skill agentdb-optimization

Installs into the current project.

Summary

Optimize AgentDB performance with quantization (4-32x memory reduction), HNSW indexing (150x faster search), caching, and batch operations. Use when optimizing memory usage, improving search speed, or scaling to millions of vectors.

Files · 1

View on GitHub

SKILL.md

1	# AgentDB Performance Optimization
2
3	## What This Skill Does
4
5	Provides comprehensive performance optimization techniques for AgentDB vector databases. Achieve 150x-12,500x performance improvements through quantization, HNSW indexing, caching strategies, and batch operations. Reduce memory usage by 4-32x while maintaining accuracy.
6
7	Performance: <100µs vector search, <1ms pattern retrieval, 2ms batch insert for 100 vectors.
8
9	## Prerequisites
10
11	- Node.js 18+
12	- AgentDB v1.0.7+ (via agentic-flow)
13	- Existing AgentDB database or application
14
15	---
16
17	## Quick Start
18
19	### Run Performance Benchmarks
20
21	```bash
22	# Comprehensive performance benchmarking
23	npx agentdb@latest benchmark
24
25	# Results show:
26	# ✅ Pattern Search: 150x faster (100µs vs 15ms)
27	# ✅ Batch Insert: 500x faster (2ms vs 1s for 100 vectors)
28	# ✅ Large-scale Query: 12,500x faster (8ms vs 100s at 1M vectors)
29	# ✅ Memory Efficiency: 4-32x reduction with quantization
30	```
31
32	### Enable Optimizations
33
34	```typescript
35	import { createAgentDBAdapter } from 'agentic-flow/reasoningbank';
36
37	// Optimized configuration
38	const adapter = await createAgentDBAdapter({
39	dbPath: '.agentdb/optimized.db',
40	quantizationType: 'binary', // 32x memory reduction
41	cacheSize: 1000, // In-memory cache
42	enableLearning: true,
43	enableReasoning: true,
44	});
45	```
46
47	---
48
49	## Quantization Strategies
50
51	### 1. Binary Quantization (32x Reduction)
52
53	Best For: Large-scale deployments (1M+ vectors), memory-constrained environments
54	Trade-off: ~2-5% accuracy loss, 32x memory reduction, 10x faster
55
56	```typescript
57	const adapter = await createAgentDBAdapter({
58	quantizationType: 'binary',
59	// 768-dim float32 (3072 bytes) → 96 bytes binary
60	// 1M vectors: 3GB → 96MB
61	});
62	```
63
64	Use Cases:
65	- Mobile/edge deployment
66	- Large-scale vector storage (millions of vectors)
67	- Real-time search with memory constraints
68
69	Performance:
70	- Memory: 32x smaller
71	- Search Speed: 10x faster (bit operations)
72	- Accuracy: 95-98% of original
73
74	### 2. Scalar Quantization (4x Reduction)
75
76	Best For: Balanced performance/accuracy, moderate datasets
77	Trade-off: ~1-2% accuracy loss, 4x memory reduction, 3x faster
78
79	```typescript
80	const adapter = await createAgentDBAdapter({
81	quantizationType: 'scalar',
82	// 768-dim float32 (3072 bytes) → 768 bytes (uint8)
83	// 1M vectors: 3GB → 768MB
84	});
85	```
86
87	Use Cases:
88	- Production applications requiring high accuracy
89	- Medium-scale deployments (10K-1M vectors)
90	- General-purpose optimization
91
92	Performance:
93	- Memory: 4x smaller
94	- Search Speed: 3x faster
95	- Accuracy: 98-99% of original
96
97	### 3. Product Quantization (8-16x Reduction)
98
99	Best For: High-dimensional vectors, balanced compression
100	Trade-off: ~3-7% accuracy loss, 8-16x memory reduction, 5x faster
101
102	```typescript
103	const adapter = await createAgentDBAdapter({
104	quantizationType: 'product',
105	// 768-dim float32 (3072 bytes) → 48-96 bytes
106	// 1M vectors: 3GB → 192MB
107	});
108	```
109
110	Use Cases:
111	- High-dimensional embeddings (>512 dims)
112	- Image/video embeddings
113	- Large-scale similarity search
114
115	Performance:
116	- Memory: 8-16x smaller
117	- Search Speed: 5x faster
118	- Accuracy: 93-97% of original
119
120	### 4. No Quantization (Full Precision)
121
122	Best For: Maximum accuracy, small datasets
123	Trade-off: No accuracy loss, full memory usage
124
125	```typescript
126	const adapter = await createAgentDBAdapter({
127	quantizationType: 'none',
128	// Full float32 precision
129	});
130	```
131
132	---
133
134	## HNSW Indexing
135
136	Hierarchical Navigable Small World - O(log n) search complexity
137
138	### Automatic HNSW
139
140	AgentDB automatically builds HNSW indices:
141
142	```typescript
143	const adapter = await createAgentDBAdapter({
144	dbPath: '.agentdb/vectors.db',
145	// HNSW automatically enabled
146	});
147
148	// Search with HNSW (100µs vs 15ms linear scan)
149	const results = await adapter.retrieveWithReasoning(queryEmbedding, {
150	k: 10,
151	});
152	```
153
154	### HNSW Parameters
155
156	```typescript
157	// Advanced HNSW configuration
158	const adapter = await createAgentDBAdapter({
159	dbPath: '.agentdb/vectors.db',
160	hnswM: 16, // Connections per layer (default: 16)
161	hnswEfConstruction: 200, // Build quality (default: 200)
162	hnswEfSearch: 100, // Search quality (default: 100)
163	});
164	```
165
166	Parameter Tuning:
167	- M (connections): Higher = better recall, more memory
168	- Small datasets (<10K): M = 8
169	- Medium datasets (10K-100K): M = 16
170	- Large datasets (>100K): M = 32
171	- efConstruction: Higher = better index quality, slower build
172	- Fast build: 100
173	- Balanced: 200 (default)
174	- High quality: 400
175	- efSearch: Higher = better recall, slower search
176	- Fast search: 50
177	- Balanced: 100 (default)
178	- High recall: 200
179
180	---
181
182	## Caching Strategies
183
184	### In-Memory Pattern Cache
185
186	```typescript
187	const adapter = await createAgent

People who liked this also viewed

6 picks

Type

agentdb-optimization

301stars24forks

View on GitHub

How to install agentdb-optimization?

spencermarx/open-code-review/agentdb-optimization

$npx -y skills add spencermarx/open-code-review --skill agentdb-optimization

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# AgentDB Performance Optimization
2
3	## What This Skill Does
4
5	Provides comprehensive performance optimization techniques for AgentDB vector databases. Achieve 150x-12,500x performance improvements through quantization, HNSW indexing, caching strategies, and batch operations. Reduce memory usage by 4-32x while maintaining accuracy.
6
7	Performance: <100µs vector search, <1ms pattern retrieval, 2ms batch insert for 100 vectors.
8
9	## Prerequisites
10
11	- Node.js 18+
12	- AgentDB v1.0.7+ (via agentic-flow)
13	- Existing AgentDB database or application
14
15	---
16
17	## Quick Start
18
19	### Run Performance Benchmarks
20
21	```bash
22	# Comprehensive performance benchmarking
23	npx agentdb@latest benchmark
24
25	# Results show:
26	# ✅ Pattern Search: 150x faster (100µs vs 15ms)
27	# ✅ Batch Insert: 500x faster (2ms vs 1s for 100 vectors)
28	# ✅ Large-scale Query: 12,500x faster (8ms vs 100s at 1M vectors)
29	# ✅ Memory Efficiency: 4-32x reduction with quantization
30	```
31
32	### Enable Optimizations
33
34	```typescript
35	import { createAgentDBAdapter } from 'agentic-flow/reasoningbank';
36
37	// Optimized configuration
38	const adapter = await createAgentDBAdapter({
39	dbPath: '.agentdb/optimized.db',
40	quantizationType: 'binary', // 32x memory reduction
41	cacheSize: 1000, // In-memory cache
42	enableLearning: true,
43	enableReasoning: true,
44	});
45	```
46
47	---
48
49	## Quantization Strategies
50
51	### 1. Binary Quantization (32x Reduction)
52
53	Best For: Large-scale deployments (1M+ vectors), memory-constrained environments
54	Trade-off: ~2-5% accuracy loss, 32x memory reduction, 10x faster
55
56	```typescript
57	const adapter = await createAgentDBAdapter({
58	quantizationType: 'binary',
59	// 768-dim float32 (3072 bytes) → 96 bytes binary
60	// 1M vectors: 3GB → 96MB
61	});
62	```
63
64	Use Cases:
65	- Mobile/edge deployment
66	- Large-scale vector storage (millions of vectors)
67	- Real-time search with memory constraints
68
69	Performance:
70	- Memory: 32x smaller
71	- Search Speed: 10x faster (bit operations)
72	- Accuracy: 95-98% of original
73
74	### 2. Scalar Quantization (4x Reduction)
75
76	Best For: Balanced performance/accuracy, moderate datasets
77	Trade-off: ~1-2% accuracy loss, 4x memory reduction, 3x faster
78
79	```typescript
80	const adapter = await createAgentDBAdapter({
81	quantizationType: 'scalar',
82	// 768-dim float32 (3072 bytes) → 768 bytes (uint8)
83	// 1M vectors: 3GB → 768MB
84	});
85	```
86
87	Use Cases:
88	- Production applications requiring high accuracy
89	- Medium-scale deployments (10K-1M vectors)
90	- General-purpose optimization
91
92	Performance:
93	- Memory: 4x smaller
94	- Search Speed: 3x faster
95	- Accuracy: 98-99% of original
96
97	### 3. Product Quantization (8-16x Reduction)
98
99	Best For: High-dimensional vectors, balanced compression
100	Trade-off: ~3-7% accuracy loss, 8-16x memory reduction, 5x faster
101
102	```typescript
103	const adapter = await createAgentDBAdapter({
104	quantizationType: 'product',
105	// 768-dim float32 (3072 bytes) → 48-96 bytes
106	// 1M vectors: 3GB → 192MB
107	});
108	```
109
110	Use Cases:
111	- High-dimensional embeddings (>512 dims)
112	- Image/video embeddings
113	- Large-scale similarity search
114
115	Performance:
116	- Memory: 8-16x smaller
117	- Search Speed: 5x faster
118	- Accuracy: 93-97% of original
119
120	### 4. No Quantization (Full Precision)
121
122	Best For: Maximum accuracy, small datasets
123	Trade-off: No accuracy loss, full memory usage
124
125	```typescript
126	const adapter = await createAgentDBAdapter({
127	quantizationType: 'none',
128	// Full float32 precision
129	});
130	```
131
132	---
133
134	## HNSW Indexing
135
136	Hierarchical Navigable Small World - O(log n) search complexity
137
138	### Automatic HNSW
139
140	AgentDB automatically builds HNSW indices:
141
142	```typescript
143	const adapter = await createAgentDBAdapter({
144	dbPath: '.agentdb/vectors.db',
145	// HNSW automatically enabled
146	});
147
148	// Search with HNSW (100µs vs 15ms linear scan)
149	const results = await adapter.retrieveWithReasoning(queryEmbedding, {
150	k: 10,
151	});
152	```
153
154	### HNSW Parameters
155
156	```typescript
157	// Advanced HNSW configuration
158	const adapter = await createAgentDBAdapter({
159	dbPath: '.agentdb/vectors.db',
160	hnswM: 16, // Connections per layer (default: 16)
161	hnswEfConstruction: 200, // Build quality (default: 200)
162	hnswEfSearch: 100, // Search quality (default: 100)
163	});
164	```
165
166	Parameter Tuning:
167	- M (connections): Higher = better recall, more memory
168	- Small datasets (<10K): M = 8
169	- Medium datasets (10K-100K): M = 16
170	- Large datasets (>100K): M = 32
171	- efConstruction: Higher = better index quality, slower build
172	- Fast build: 100
173	- Balanced: 200 (default)
174	- High quality: 400
175	- efSearch: Higher = better recall, slower search
176	- Fast search: 50
177	- Balanced: 100 (default)
178	- High recall: 200
179
180	---
181
182	## Caching Strategies
183
184	### In-Memory Pattern Cache
185
186	```typescript
187	const adapter = await createAgent

People who liked this also viewed

6 picks

Type