Skip to content
Compresr docs

Introduction

Compresr shrinks the context you send to your LLM without losing the answer.

Compresr is a context-compression API for LLM developers. Send the long text you'd pass to your model with the query you want answered; Compresr keeps the spans that carry the answer, drops the rest, and returns a shorter context you forward to your LLM. The result: fewer input tokens, lower cost, a longer effective context window, and faster inference. It sits in front of whatever LLM stack you already use and replaces nothing.

Use cases

Anywhere you send long text with a question about it:

  • RAG — compress retrieved chunks before they reach the model, so you only pay for tokens that carry the information.
  • Conversations — compress growing chat history so long sessions stay inside the context window.
  • Tool outputs (agents) — compress noisy tool results — web search hits, API responses, file dumps — before they re-enter the prompt. Pass the tool call's intent as the query so Compresr keeps only what the agent asked for.
  • Document understanding — ask a question against long, dense documents — legal contracts, medical records, financial filings — and keep only the spans that answer it.

Models and parameters

Three query-specific compression models are available: latte_v1 (battle-tested, stable), latte_v2 (beta — faster, served at 2× the base rate limits, and adds a dynamic mode that picks the compression ratio per input), and latte_v2.1 (beta — the newest checkpoint on the latte_v2 stack: same speed, same price, same parameters, trained on a broader mix). latte_v2 is a drop-in replacement for latte_v1; reach for it by default.

Every call takes three required fields — the context to compress, the query to score it against, and a compression_model_name — plus optional knobs like target_compression_ratio and coarse. The full parameter matrix, defaults, and ratio semantics live in Models & parameters.

Start with

Pick a language

The shape of the call is identical across all three; only the syntax differs.

  • Python - pip install compresr, then call client.compress(...).
  • TypeScript - npm install @compresr/sdk, then call client.compress({...}).
  • cURL - one POST request, no install required.

Framework integrations

First-party integrations ship in both SDKs.

  • LangChain: three middlewares (tool output, history summarization, prompt budget) + RAG document compressor + single-tool wrapper (HOF or decorator), for create_agent and ContextualCompressionRetriever.
  • LangGraph: re-exports the three middlewares plus adds make_compresr_node, CompresrCheckpointSerializer, CompresrStore, and compresr_handoff_tool for state graphs, at-rest compression, and supervisor → sub-agent transfers.
  • LlamaIndex: CompresrNodePostprocessor for query engines, wrap_tool_with_compresr for FunctionTools, and CompresrMemoryBlock for the Memory API.
  • LiteLLM: Python-only pre_call guardrail that auto-compresses tool/function messages before they go upstream, working against every LiteLLM provider.
  • LLM provider recipes: manual pattern called directly against OpenAI, Anthropic, Gemini, or local Ollama.

Related reading