Skip to content
Compresr docs

Guides

Query-specific compression

Send a query with your context so the model keeps the parts that answer it and drops the rest.

Query-specific compression shortens your context while keeping the parts that answer your query. You send both; the API drops everything irrelevant. Both latte_v1 and latte_v2 support this. Both require a non-empty query; the API rejects an empty or missing one with 422.

SDK default is latte_v1

The SDK uses latte_v1 unless you set compression_model_name. Samples on this page pass compression_model_name="latte_v2" explicitly. Drop that argument and you're back on latte_v1.

What query-specific means

With a query, you tell the compressor what to keep instead of letting it guess. A query-agnostic compressor keeps whatever looks generally interesting. The latte models instead score spans of the context against your query and keep the ones with the highest signal. You usually already know what the next step does with the context: answer a question, extract a field, summarize against a goal. That knowledge is what the compressor needs.

The query is not a prompt the model answers. It only steers what gets kept. The output is still your original context, just shorter.

What makes a good query

Write the query like a one-line task description for the next step. Concrete beats vague. It should tell the model what to preserve.

  • Good: "What was the project's Q3 churn rate?"
  • Good: "Find the renewal date in the customer contract."
  • Good: "Summarize the customer's stated reasons for canceling."
  • Weak: "churn" - single keyword, no intent
  • Weak: "important info" - no signal at all
  • Rejected: "" - both SDK schemas enforce min_length=1 and raise ValidationError before the request goes out. To skip query-aware behavior on latte_v2, omit the field entirely. In Python, don't pass query=; in TS, don't set query:. On latte_v1, omitting query fails server-side with a 422 that the SDK maps to ValidationError.

The query does not need a question mark. The last user message in a chat, the current sub-goal in an agent loop, or the task summary in a job spec all work.

A working example

The input below is an earnings summary covering several figures. The query asks for one: the full-year guidance. The compressor keeps the guidance sentence and drops the rest.

Response nullability

On error, result.data is None in Python / null in TypeScript. Check it before reading compressed_context. The samples below skip that check for readability.

python

Exact numbers vary by run.

Tuning compression strength

Set target_compression_ratio to control how much the model cuts. Values in 0 < r <= 1 are removal-strength; values > 1 are Nx-factor mode. See the Models reference for full semantics, defaults, and bounds.

python

On the earnings summary, 0.5 keeps supporting figures alongside the guidance sentence — the revenue growth, the buyback. That suits answers that need some grounding. A setting like 4 targets roughly a quarter of the original and collapses the output to the guidance sentence itself. That suits extractive Q&A with tight token budgets.

Truncation slices off whatever happens to be at the end. Query-specific compression instead keeps the coherent spans that answer the query, wherever they sit in the document.

Building a RAG pipeline?

The RAG guide shows the full retrieve, compress, and LLM flow with LangChain, LlamaIndex, and direct vector-DB examples.