context-compression-experiments-2508

Name: context-compression-experiments-2508
Author: Laurian

70stars6forks

How to install context-compression-experiments-2508?

laurian/context-compression-experiments-2508

$git clone https://github.com/Laurian/context-compression-experiments-2508

Installs into the current project.

Summary

I have in production an Open Deep Research-like Agentic RAG over an internal knowledge base. Upon retrieval each document needs context compression: extracting the content that is relevant to the query that retrieved the document, extracting paragraphs by semantic similarity is n

Files · 1

View on GitHub

README.md

1	# Context Compression Prompt Experiments
2
3	I have in production an Open Deep Research-like Agentic RAG over an internal knowledge base. Upon retrieval each document needs context compression: extracting the content that is relevant to the query that retrieved the document, extracting paragraphs by semantic similarity is not good enough and a LLM is way more accurate.
4
5	The prompt (inspired from the LangChain's [LLMChainExtractor](https://python.langchain.com/api_reference/langchain/retrievers/langchain.retrievers.document_compressors.chain_extract.LLMChainExtractor.html) [prompt](https://github.com/langchain-ai/langchain/blob/b999f356e86b706d943ac2c3ee9b21a0cffeefa5/libs/langchain/langchain/retrievers/document_compressors/chain_extract_prompt.py)) I use is:
6
7	```md
8	You are tasked with performing a contextual compression of a document as part of a system that processes multiple documents. Your goal is to extract only the essential parts of the given context that are relevant to a specific query.
9	This process helps in focusing on the most important information and reducing noise in the context.
10	The query might refer to multiple documents, consider how does apply to a single document in the context as multiple documents might be relevant.
11
12	Your task is to extract any parts of the context that are directly relevant to answering this question. Follow these guidelines:
13
14	1. Only extract text AS IS that is directly related to the query.
15	2. Do not modify, paraphrase, or summarize the extracted text. Copy it exactly as it appears in the context.
16	3. You may extract multiple separate parts if necessary.
17	4. If a header relates to the query, extract also the text under that section.
18	5. Preserve headings and subheadings when extracting.
19	6. If you find no relevant information in the context, output "NO_OUTPUT".
20
21	Here is the context document:
22
23	<context>[MARKDOWN_CONTENT]</context>
24
25	Now, consider the following query:
26
27	<query>[EXAMPLE_QUERY]</query>
28
29	Now, proceed with the task using the provided context and query.
30	```
31
32	Which performs well with `gpt-4o` and not that great with `gpt-4o-mini` and in production we frequently fallback to `gpt-4o-mini` due to rate limits on Azure OpenAI.
33
34	I filtered out 1000+ `gpt-4o-mini` traces from LangFuse that failed contextual compressions (nothing extracted from the document: `NO_OUTPUT`) and re-ran them against `gpt-4o` which was successful in about 296 of them. That's my sample dataset of document × query pairs for which I would like to make `gpt-4o-mini` perform better.
35
36	Please note: Certain domain-specific details have been intentionally omitted for privacy and confidentiality.
37
38
39	## DSPy GEPA (Genetic Pareto)
40
41	I first used [DSPy](https://dspy.ai/api/optimizers/GEPA/) [GEPA (Genetic-Pareto)](https://github.com/gepa-ai/gepa) to optimise the prompt, basically had Claude Code understand the input dataset and after a couple of iterations on some errors and misunderstandings of the goal, I got it running.
42
43
44	Running [dspy_gepa_optimizer.py](scripts/dspy_gepa_optimizer.py):
45
46	```md
47	Running DSPy GEPA optimization...
48	This will optimize the context compression prompt using genetic algorithms.
49	Make sure you have configured your .env file with API keys.
50	uv run python scripts/dspy_gepa_optimizer.py
51	INFO:__main__:Loading observation data...
52	INFO:__main__:Loaded 296 valid observations
53	INFO:__main__:Observations with GPT-4o targets: 296
54	INFO:__main__:DSPy configured with GPT-4o-mini, cache dir: cache/dspy
55	INFO:__main__:WANDB_API_KEY not found or commented out, skipping Weave initialization
56	WARNING:weave.trace.op:Warning: Traces will not be logged. Call weave.init to log your traces to a project.
57	(subsequent messages of this type will be suppressed)
58	INFO:__main__:Prepared 50 examples for optimization
59	INFO:__main__:Training on 40 examples, validating on 10
60	INFO:__main__:Initialized ContextCompressor with base prompt: You are tasked with performing a contextual compression of a document as part of a system that proce...
61	I

People who liked this also viewed

6 picks

Type

context-compression-experiments-2508

70stars6forks

View on GitHub

How to install context-compression-experiments-2508?

laurian/context-compression-experiments-2508

$git clone https://github.com/Laurian/context-compression-experiments-2508

Installs into the current project.

Summary

Files · 1

View on GitHub

README.md

1	# Context Compression Prompt Experiments
2
3	I have in production an Open Deep Research-like Agentic RAG over an internal knowledge base. Upon retrieval each document needs context compression: extracting the content that is relevant to the query that retrieved the document, extracting paragraphs by semantic similarity is not good enough and a LLM is way more accurate.
4
5	The prompt (inspired from the LangChain's [LLMChainExtractor](https://python.langchain.com/api_reference/langchain/retrievers/langchain.retrievers.document_compressors.chain_extract.LLMChainExtractor.html) [prompt](https://github.com/langchain-ai/langchain/blob/b999f356e86b706d943ac2c3ee9b21a0cffeefa5/libs/langchain/langchain/retrievers/document_compressors/chain_extract_prompt.py)) I use is:
6
7	```md
8	You are tasked with performing a contextual compression of a document as part of a system that processes multiple documents. Your goal is to extract only the essential parts of the given context that are relevant to a specific query.
9	This process helps in focusing on the most important information and reducing noise in the context.
10	The query might refer to multiple documents, consider how does apply to a single document in the context as multiple documents might be relevant.
11
12	Your task is to extract any parts of the context that are directly relevant to answering this question. Follow these guidelines:
13
14	1. Only extract text AS IS that is directly related to the query.
15	2. Do not modify, paraphrase, or summarize the extracted text. Copy it exactly as it appears in the context.
16	3. You may extract multiple separate parts if necessary.
17	4. If a header relates to the query, extract also the text under that section.
18	5. Preserve headings and subheadings when extracting.
19	6. If you find no relevant information in the context, output "NO_OUTPUT".
20
21	Here is the context document:
22
23	<context>[MARKDOWN_CONTENT]</context>
24
25	Now, consider the following query:
26
27	<query>[EXAMPLE_QUERY]</query>
28
29	Now, proceed with the task using the provided context and query.
30	```
31
32	Which performs well with `gpt-4o` and not that great with `gpt-4o-mini` and in production we frequently fallback to `gpt-4o-mini` due to rate limits on Azure OpenAI.
33
34	I filtered out 1000+ `gpt-4o-mini` traces from LangFuse that failed contextual compressions (nothing extracted from the document: `NO_OUTPUT`) and re-ran them against `gpt-4o` which was successful in about 296 of them. That's my sample dataset of document × query pairs for which I would like to make `gpt-4o-mini` perform better.
35
36	Please note: Certain domain-specific details have been intentionally omitted for privacy and confidentiality.
37
38
39	## DSPy GEPA (Genetic Pareto)
40
41	I first used [DSPy](https://dspy.ai/api/optimizers/GEPA/) [GEPA (Genetic-Pareto)](https://github.com/gepa-ai/gepa) to optimise the prompt, basically had Claude Code understand the input dataset and after a couple of iterations on some errors and misunderstandings of the goal, I got it running.
42
43
44	Running [dspy_gepa_optimizer.py](scripts/dspy_gepa_optimizer.py):
45
46	```md
47	Running DSPy GEPA optimization...
48	This will optimize the context compression prompt using genetic algorithms.
49	Make sure you have configured your .env file with API keys.
50	uv run python scripts/dspy_gepa_optimizer.py
51	INFO:__main__:Loading observation data...
52	INFO:__main__:Loaded 296 valid observations
53	INFO:__main__:Observations with GPT-4o targets: 296
54	INFO:__main__:DSPy configured with GPT-4o-mini, cache dir: cache/dspy
55	INFO:__main__:WANDB_API_KEY not found or commented out, skipping Weave initialization
56	WARNING:weave.trace.op:Warning: Traces will not be logged. Call weave.init to log your traces to a project.
57	(subsequent messages of this type will be suppressed)
58	INFO:__main__:Prepared 50 examples for optimization
59	INFO:__main__:Training on 40 examples, validating on 10
60	INFO:__main__:Initialized ContextCompressor with base prompt: You are tasked with performing a contextual compression of a document as part of a system that proce...
61	I

People who liked this also viewed

6 picks

Type