content-hash-cache-pattern

Name: content-hash-cache-pattern
Author: affaan-m

232kstars35kforks

How to install content-hash-cache-pattern?

affaan-m/ecc/content-hash-cache-pattern

$npx -y skills add affaan-m/ECC --skill content-hash-cache-pattern

Installs into the current project.

Summary

Cache expensive file processing results using SHA-256 content hashes — path-independent, auto-invalidating, with service layer separation.

Files · 1

View on GitHub

SKILL.md

1	# Content-Hash File Cache Pattern
2
3	Cache expensive file processing results (PDF parsing, text extraction, image analysis) using SHA-256 content hashes as cache keys. Unlike path-based caching, this approach survives file moves/renames and auto-invalidates when content changes.
4
5	## When to Activate
6
7	- Building file processing pipelines (PDF, images, text extraction)
8	- Processing cost is high and same files are processed repeatedly
9	- Need a `--cache/--no-cache` CLI option
10	- Want to add caching to existing pure functions without modifying them
11
12	## Core Pattern
13
14	### 1. Content-Hash-Based Cache Key
15
16	Use file content (not path) as the cache key:
17
18	```python
19	import hashlib
20	from pathlib import Path
21
22	_HASH_CHUNK_SIZE = 65536 # 64KB chunks for large files
23
24	def compute_file_hash(path: Path) -> str:
25	"""SHA-256 of file contents (chunked for large files)."""
26	if not path.is_file():
27	raise FileNotFoundError(f"File not found: {path}")
28	sha256 = hashlib.sha256()
29	with open(path, "rb") as f:
30	while True:
31	chunk = f.read(_HASH_CHUNK_SIZE)
32	if not chunk:
33	break
34	sha256.update(chunk)
35	return sha256.hexdigest()
36	```
37
38	Why content hash? File rename/move = cache hit. Content change = automatic invalidation. No index file needed.
39
40	### 2. Frozen Dataclass for Cache Entry
41
42	```python
43	from dataclasses import dataclass
44
45	@dataclass(frozen=True, slots=True)
46	class CacheEntry:
47	file_hash: str
48	source_path: str
49	document: ExtractedDocument # The cached result
50	```
51
52	### 3. File-Based Cache Storage
53
54	Each cache entry is stored as `{hash}.json` — O(1) lookup by hash, no index file required.
55
56	```python
57	import json
58	from typing import Any
59
60	def write_cache(cache_dir: Path, entry: CacheEntry) -> None:
61	cache_dir.mkdir(parents=True, exist_ok=True)
62	cache_file = cache_dir / f"{entry.file_hash}.json"
63	data = serialize_entry(entry)
64	cache_file.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
65
66	def read_cache(cache_dir: Path, file_hash: str) -> CacheEntry \| None:
67	cache_file = cache_dir / f"{file_hash}.json"
68	if not cache_file.is_file():
69	return None
70	try:
71	raw = cache_file.read_text(encoding="utf-8")
72	data = json.loads(raw)
73	return deserialize_entry(data)
74	except (json.JSONDecodeError, ValueError, KeyError):
75	return None # Treat corruption as cache miss
76	```
77
78	### 4. Service Layer Wrapper (SRP)
79
80	Keep the processing function pure. Add caching as a separate service layer.
81
82	```python
83	def extract_with_cache(
84	file_path: Path,
85	*,
86	cache_enabled: bool = True,
87	cache_dir: Path = Path(".cache"),
88	) -> ExtractedDocument:
89	"""Service layer: cache check -> extraction -> cache write."""
90	if not cache_enabled:
91	return extract_text(file_path) # Pure function, no cache knowledge
92
93	file_hash = compute_file_hash(file_path)
94
95	# Check cache
96	cached = read_cache(cache_dir, file_hash)
97	if cached is not None:
98	logger.info("Cache hit: %s (hash=%s)", file_path.name, file_hash[:12])
99	return cached.document
100
101	# Cache miss -> extract -> store
102	logger.info("Cache miss: %s (hash=%s)", file_path.name, file_hash[:12])
103	doc = extract_text(file_path)
104	entry = CacheEntry(file_hash=file_hash, source_path=str(file_path), document=doc)
105	write_cache(cache_dir, entry)
106	return doc
107	```
108
109	## Key Design Decisions
110
111	\| Decision \| Rationale \|
112	\|----------\|-----------\|
113	\| SHA-256 content hash \| Path-independent, auto-invalidates on content change \|
114	\| `{hash}.json` file naming \| O(1) lookup, no index file needed \|
115	\| Service layer wrapper \| SRP: extraction stays pure, cache is a separate concern \|
116	\| Manual JSON serialization \| Full control over frozen dataclass serialization \|
117	\| Corruption returns `None` \| Graceful degradation, re-processes on next run \|
118	\| `cache_dir.mkdir(parents=True)` \| Lazy directory creation on first write \|
119
120	## Best Practices
121
122	- Hash content, not paths — paths change, content identity doesn't
123	- Chunk large files when hashing — avoid loading entire files into memory
124	- Keep processing functions pure — they should know nothing about caching
125	- Log cache hit/miss with truncated hashes for debugging
126	- Handle corruption gracefully — treat invalid cache entries as misses, never crash
127
128	## Anti-Patterns to Avoid
129
130	```python
131	# BAD: Path-based caching (breaks on file move/rename)
132	cache = {"/path/to/file.pdf": result}
133
134	# BAD: Adding cache logic inside the processing function (SRP violation)
135	def extract_text(path, *, cache_enabled=False, cache_dir=None):
136	if cache_enabled: # Now this function has two responsibilities
137	...
138
139	# BAD: Using dataclasses.asdict() with nested frozen dataclasses
140	# (can cause issues with complex nested types)
141	dat

People who liked this also viewed

6 picks

Type

find-skillsHelps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skSkillsJul 20262.6M27k
grill-meA relentless interview to sharpen a plan or design.SkillsJul 2026633k182k
vercel-react-best-practicesVercel's official collection of agent skillsSkillsJul 2026573k29k
improve-codebase-architectureScan a codebase for deepening opportunities, present them as a visual HTML report, then grill through whichever one you pick.SkillsJul 2026522k182k
setup-matt-pocock-skillsConfigure this repo for the engineering skills — set up its issue tracker, triage label vocabulary, and domain doc layout. Run once before first use of the other engineering skills.SkillsJul 2026454k182k
triageMove issues and external PRs through a state machine of triage roles — categorise, verify, grill if needed, and write agent-ready briefs.SkillsJul 2026424k182k

content-hash-cache-pattern

232kstars35kforks

View on GitHub

How to install content-hash-cache-pattern?

affaan-m/ecc/content-hash-cache-pattern

$npx -y skills add affaan-m/ECC --skill content-hash-cache-pattern

Installs into the current project.

Summary

Cache expensive file processing results using SHA-256 content hashes — path-independent, auto-invalidating, with service layer separation.

Files · 1

View on GitHub

SKILL.md

1	# Content-Hash File Cache Pattern
2
3	Cache expensive file processing results (PDF parsing, text extraction, image analysis) using SHA-256 content hashes as cache keys. Unlike path-based caching, this approach survives file moves/renames and auto-invalidates when content changes.
4
5	## When to Activate
6
7	- Building file processing pipelines (PDF, images, text extraction)
8	- Processing cost is high and same files are processed repeatedly
9	- Need a `--cache/--no-cache` CLI option
10	- Want to add caching to existing pure functions without modifying them
11
12	## Core Pattern
13
14	### 1. Content-Hash-Based Cache Key
15
16	Use file content (not path) as the cache key:
17
18	```python
19	import hashlib
20	from pathlib import Path
21
22	_HASH_CHUNK_SIZE = 65536 # 64KB chunks for large files
23
24	def compute_file_hash(path: Path) -> str:
25	"""SHA-256 of file contents (chunked for large files)."""
26	if not path.is_file():
27	raise FileNotFoundError(f"File not found: {path}")
28	sha256 = hashlib.sha256()
29	with open(path, "rb") as f:
30	while True:
31	chunk = f.read(_HASH_CHUNK_SIZE)
32	if not chunk:
33	break
34	sha256.update(chunk)
35	return sha256.hexdigest()
36	```
37
38	Why content hash? File rename/move = cache hit. Content change = automatic invalidation. No index file needed.
39
40	### 2. Frozen Dataclass for Cache Entry
41
42	```python
43	from dataclasses import dataclass
44
45	@dataclass(frozen=True, slots=True)
46	class CacheEntry:
47	file_hash: str
48	source_path: str
49	document: ExtractedDocument # The cached result
50	```
51
52	### 3. File-Based Cache Storage
53
54	Each cache entry is stored as `{hash}.json` — O(1) lookup by hash, no index file required.
55
56	```python
57	import json
58	from typing import Any
59
60	def write_cache(cache_dir: Path, entry: CacheEntry) -> None:
61	cache_dir.mkdir(parents=True, exist_ok=True)
62	cache_file = cache_dir / f"{entry.file_hash}.json"
63	data = serialize_entry(entry)
64	cache_file.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
65
66	def read_cache(cache_dir: Path, file_hash: str) -> CacheEntry \| None:
67	cache_file = cache_dir / f"{file_hash}.json"
68	if not cache_file.is_file():
69	return None
70	try:
71	raw = cache_file.read_text(encoding="utf-8")
72	data = json.loads(raw)
73	return deserialize_entry(data)
74	except (json.JSONDecodeError, ValueError, KeyError):
75	return None # Treat corruption as cache miss
76	```
77
78	### 4. Service Layer Wrapper (SRP)
79
80	Keep the processing function pure. Add caching as a separate service layer.
81
82	```python
83	def extract_with_cache(
84	file_path: Path,
85	*,
86	cache_enabled: bool = True,
87	cache_dir: Path = Path(".cache"),
88	) -> ExtractedDocument:
89	"""Service layer: cache check -> extraction -> cache write."""
90	if not cache_enabled:
91	return extract_text(file_path) # Pure function, no cache knowledge
92
93	file_hash = compute_file_hash(file_path)
94
95	# Check cache
96	cached = read_cache(cache_dir, file_hash)
97	if cached is not None:
98	logger.info("Cache hit: %s (hash=%s)", file_path.name, file_hash[:12])
99	return cached.document
100
101	# Cache miss -> extract -> store
102	logger.info("Cache miss: %s (hash=%s)", file_path.name, file_hash[:12])
103	doc = extract_text(file_path)
104	entry = CacheEntry(file_hash=file_hash, source_path=str(file_path), document=doc)
105	write_cache(cache_dir, entry)
106	return doc
107	```
108
109	## Key Design Decisions
110
111	\| Decision \| Rationale \|
112	\|----------\|-----------\|
113	\| SHA-256 content hash \| Path-independent, auto-invalidates on content change \|
114	\| `{hash}.json` file naming \| O(1) lookup, no index file needed \|
115	\| Service layer wrapper \| SRP: extraction stays pure, cache is a separate concern \|
116	\| Manual JSON serialization \| Full control over frozen dataclass serialization \|
117	\| Corruption returns `None` \| Graceful degradation, re-processes on next run \|
118	\| `cache_dir.mkdir(parents=True)` \| Lazy directory creation on first write \|
119
120	## Best Practices
121
122	- Hash content, not paths — paths change, content identity doesn't
123	- Chunk large files when hashing — avoid loading entire files into memory
124	- Keep processing functions pure — they should know nothing about caching
125	- Log cache hit/miss with truncated hashes for debugging
126	- Handle corruption gracefully — treat invalid cache entries as misses, never crash
127
128	## Anti-Patterns to Avoid
129
130	```python
131	# BAD: Path-based caching (breaks on file move/rename)
132	cache = {"/path/to/file.pdf": result}
133
134	# BAD: Adding cache logic inside the processing function (SRP violation)
135	def extract_text(path, *, cache_enabled=False, cache_dir=None):
136	if cache_enabled: # Now this function has two responsibilities
137	...
138
139	# BAD: Using dataclasses.asdict() with nested frozen dataclasses
140	# (can cause issues with complex nested types)
141	dat

People who liked this also viewed

6 picks

Type