LlamaIndex
First-party Compresr node postprocessor, tool wrapper, and memory block for LlamaIndex query engines and chat agents for Python and TypeScript.
compresr.integrations.llamaindex (Python) and @compresr/sdk/integrations/llamaindex (TypeScript) cover the three places LlamaIndex users typically burn tokens: retrieved nodes feeding a query engine, verbose FunctionTool outputs in an agent, and long-running chat history under the Memory API. Every class is a drop-in subclass of the canonical LlamaIndex base, so they slot into existing pipelines without touching the rest of your graph.
1. Install
2. Compress retrieved nodes (query engines)
CompresrNodePostprocessor is a BaseNodePostprocessor; pass it to as_query_engine or any RetrieverQueryEngine and the retrieved nodes are compressed query-aware before synthesis ever sees them — retrieve aggressively (similarity_top_k=20+) and let compression keep the synthesis prompt tight. See the RAG guide for the surrounding pipeline.
The postprocessor copies each NodeWithScore (originals are untouched in the index), partitions by min_tokens, batches the eligible nodes into slices of 100 and issues one Compresr call per slice, and writes new text on the copies via node.set_content() (falling back to node.text = and finally to metadata["compresr_compressed"] with a warning). If the query can't be resolved, it logs a warning and passes nodes through unchanged: query is required by both models, so no call is made without one.
Options
| Python | TypeScript | Default | Notes |
|---|---|---|---|
compression_model | compressionModel | "latte_v1" | See Models. |
target_compression_ratio | targetCompressionRatio | 0.5 | 0 < r ≤ 1 removal fraction; r > 1 Nx target. |
target_token | targetToken | n/a | Alternative: absolute output budget per node. Overrides ratio (ratio = max(avg_chunk_tokens / target_token, 1.0)). When target_token > current tokens the ratio pins to 1.0 (no compression is applied). |
min_tokens | minTokens | 200 | Skip nodes shorter than this. Estimator uses tiktoken when available; otherwise falls back to chars/4. |
coarse | coarse | None | When None, defers to backend default (paragraph-level). |
query | query | n/a | Override the query string (otherwise pulled from QueryBundle.query_str). |
on_error | onError | "passthrough" | Fail-open by default. |
api_key / base_url / client | apiKey / baseUrl / client | n/a | Standard auth knobs. |
3. Compress an agent tool's output
wrap_tool_with_compresr / wrapToolWithCompresr takes a FunctionTool (or tool-like duck type in TypeScript) and returns a new one whose return value is compressed transparently. The Python version preserves the original tool's name, description, fn_schema, and return_direct via FunctionTool.from_defaults(...).
If the wrapped function returns anything other than a string, it's passed through unchanged. Python's wrapper also automatically wires an async branch when the source tool exposes async_fn. The TypeScript wrapper currently overrides only .call — tools that expose .acall bypass compression on the async path.
Query resolution: static query wins if set; otherwise query_extractor(args) is called if provided; otherwise, and only if you set neither, the wrapper consults args[query_arg] when query_arg is set (strict: if the named key is missing, the resolver returns None and does NOT fall back), or smart-picks from common keys (query, question, search_query, q, prompt, input, text) when query_arg is unset.
4. Compress chat history (Memory API)
CompresrMemoryBlock is a BaseMemoryBlock[str]; register it on Memory.from_defaults and the long-running buffer is compressed via Compresr when the memory layer needs to free tokens — no extra LLM-summary call.
Python and TypeScript compress at different moments
Python compresses inside atruncate(), the Memory layer's truncation hook. TypeScript compresses inside get(), since LlamaIndex.TS's BaseMemoryBlock has no atruncate. The observable output (a single compressed system message containing the history) is identical.
When the buffer overflows, the block calls Compresr with latte_v1, using the last user: line of the buffer as the query (or "conversation history" if no user line exists). Override with query= for a fixed query string, or set target_token for a target output-token budget (best-effort — translated to a compression ratio, not a hard cap). Other fields: name (default "compresr_compressed_history"), priority (default 2), target_compression_ratio (default 0.5), coarse (default None), on_error (default "passthrough").