redis-semantic-cache

Name: redis-semantic-cache
Author: redis

774installs22forks

How to install redis-semantic-cache?

redis/agent-skills/redis-semantic-cache

$npx -y skills add redis/agent-skills --skill redis-semantic-cache

Installs into the current project.

Summary

Redis LangCache guidance for semantic caching of LLM responses on Redis Cloud — calling search/set via the SDK or REST API, tuning the similarity threshold, separating caches per task type, and filtering with custom attributes. Use when caching LLM completions or RAG answers to c

Files · 1

View on GitHub

SKILL.md

1	# Redis Semantic Cache
2
3	Semantic caching for LLM responses with Redis Cloud's LangCache service. Stores prompts as embeddings; subsequent semantically-similar prompts return the cached response without re-calling the model.
4
5	> LangCache is currently in preview on Redis Cloud. Features and behavior may change.
6
7	## When to apply
8
9	- Wrapping an LLM call (OpenAI, Anthropic, etc.) with a cache layer to cut cost and latency.
10	- Caching RAG answers, classification outputs, or any deterministic LLM workload.
11	- Tuning the precision/hit-rate trade-off for a semantic cache.
12	- Splitting one application's LLM workloads across multiple cache instances.
13
14	## 1. The cache-aside flow
15
16	LangCache fits in front of any LLM call as a standard cache-aside pattern:
17
18	1. Send the user's prompt to LangCache's `search`.
19	2. Cache hit — return the stored response directly.
20	3. Cache miss — call the LLM, then `set` the response so future similar prompts hit.
21
22	```python
23	from langcache import LangCache
24	import os
25
26	lang_cache = LangCache(
27	server_url=f"https://{os.getenv('HOST')}",
28	cache_id=os.getenv("CACHE_ID"),
29	api_key=os.getenv("API_KEY"),
30	)
31
32	result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.9)
33	if result:
34	response = result[0]["response"]
35	else:
36	response = llm.generate("What is Redis?")
37	lang_cache.set(prompt="What is Redis?", response=response)
38	```
39
40	The same operations are available via REST (`POST /v1/caches/{cacheId}/entries/search` and `POST /v1/caches/{cacheId}/entries`) when an SDK isn't an option.
41
42	See [references/langcache-usage.md](references/langcache-usage.md) for full SDK + REST samples and attribute-based storage.
43
44	## 2. Tune the similarity threshold
45
46	The threshold controls how close (in embedding cosine distance) a new prompt must be to a cached one to count as a hit. Higher = stricter match, fewer false positives. Lower = more hits, more risk of returning an off-topic answer.
47
48	\| Threshold \| Behavior \| Use when \|
49	\|---\|---\|---\|
50	\| 0.95+ \| Near-exact match required \| Customer-facing answers where wrong responses are costly \|
51	\| 0.9 \| Balanced default \| Most workloads — start here \|
52	\| 0.8 \| Loose semantic match \| Internal tools, exploratory queries, FAQ deduplication \|
53
54	```python
55	# Stricter — fewer false positives
56	result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.95)
57
58	# Looser — higher hit rate
59	result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.8)
60	```
61
62	Adjust by watching the actual cache-hit rate and spot-checking that returned answers are still relevant.
63
64	See [references/best-practices.md](references/best-practices.md).
65
66	## 3. Separate caches per task type
67
68	Different LLM workloads should not share one cache — a "code question" prompt is semantically close to other code questions but has nothing to do with a password-reset support query, and crossing them returns garbage.
69
70	```python
71	support_cache = LangCache(server_url=..., cache_id="support-cache-id", api_key=...)
72	code_cache = LangCache(server_url=..., cache_id="code-cache-id", api_key=...)
73	```
74
75	Create distinct cache IDs in Redis Cloud per task, and route each call to the right one. As a finer-grained alternative, store and search with custom attributes (e.g. `{"category": "database"}`) to keep tasks in the same cache but isolated by attribute filter — useful when the same prompt format spans subtopics.
76
77	## References
78
79	- [LangCache documentation](https://redis.io/docs/latest/develop/ai/langcache/)

People who liked this also viewed

6 picks

Type

redis-semantic-cache

774installs22forks

View on GitHub

How to install redis-semantic-cache?

redis/agent-skills/redis-semantic-cache

$npx -y skills add redis/agent-skills --skill redis-semantic-cache

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Redis Semantic Cache
2
3	Semantic caching for LLM responses with Redis Cloud's LangCache service. Stores prompts as embeddings; subsequent semantically-similar prompts return the cached response without re-calling the model.
4
5	> LangCache is currently in preview on Redis Cloud. Features and behavior may change.
6
7	## When to apply
8
9	- Wrapping an LLM call (OpenAI, Anthropic, etc.) with a cache layer to cut cost and latency.
10	- Caching RAG answers, classification outputs, or any deterministic LLM workload.
11	- Tuning the precision/hit-rate trade-off for a semantic cache.
12	- Splitting one application's LLM workloads across multiple cache instances.
13
14	## 1. The cache-aside flow
15
16	LangCache fits in front of any LLM call as a standard cache-aside pattern:
17
18	1. Send the user's prompt to LangCache's `search`.
19	2. Cache hit — return the stored response directly.
20	3. Cache miss — call the LLM, then `set` the response so future similar prompts hit.
21
22	```python
23	from langcache import LangCache
24	import os
25
26	lang_cache = LangCache(
27	server_url=f"https://{os.getenv('HOST')}",
28	cache_id=os.getenv("CACHE_ID"),
29	api_key=os.getenv("API_KEY"),
30	)
31
32	result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.9)
33	if result:
34	response = result[0]["response"]
35	else:
36	response = llm.generate("What is Redis?")
37	lang_cache.set(prompt="What is Redis?", response=response)
38	```
39
40	The same operations are available via REST (`POST /v1/caches/{cacheId}/entries/search` and `POST /v1/caches/{cacheId}/entries`) when an SDK isn't an option.
41
42	See [references/langcache-usage.md](references/langcache-usage.md) for full SDK + REST samples and attribute-based storage.
43
44	## 2. Tune the similarity threshold
45
46	The threshold controls how close (in embedding cosine distance) a new prompt must be to a cached one to count as a hit. Higher = stricter match, fewer false positives. Lower = more hits, more risk of returning an off-topic answer.
47
48	\| Threshold \| Behavior \| Use when \|
49	\|---\|---\|---\|
50	\| 0.95+ \| Near-exact match required \| Customer-facing answers where wrong responses are costly \|
51	\| 0.9 \| Balanced default \| Most workloads — start here \|
52	\| 0.8 \| Loose semantic match \| Internal tools, exploratory queries, FAQ deduplication \|
53
54	```python
55	# Stricter — fewer false positives
56	result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.95)
57
58	# Looser — higher hit rate
59	result = lang_cache.search(prompt="What is Redis?", similarity_threshold=0.8)
60	```
61
62	Adjust by watching the actual cache-hit rate and spot-checking that returned answers are still relevant.
63
64	See [references/best-practices.md](references/best-practices.md).
65
66	## 3. Separate caches per task type
67
68	Different LLM workloads should not share one cache — a "code question" prompt is semantically close to other code questions but has nothing to do with a password-reset support query, and crossing them returns garbage.
69
70	```python
71	support_cache = LangCache(server_url=..., cache_id="support-cache-id", api_key=...)
72	code_cache = LangCache(server_url=..., cache_id="code-cache-id", api_key=...)
73	```
74
75	Create distinct cache IDs in Redis Cloud per task, and route each call to the right one. As a finer-grained alternative, store and search with custom attributes (e.g. `{"category": "database"}`) to keep tasks in the same cache but isolated by attribute filter — useful when the same prompt format spans subtopics.
76
77	## References
78
79	- [LangCache documentation](https://redis.io/docs/latest/develop/ai/langcache/)

People who liked this also viewed

6 picks

Type