Coarse mode

You can trade precision for speed with the coarse flag. Both latte_v1 and latte_v2 score at paragraph granularity by default: that's coarse mode. Set coarse=false for the slower, more precise token-level pass.

How coarse vs fine compares

In coarse mode the model keeps or drops whole paragraphs based on how relevant each one is to your query. Setting coarse=false switches to token-level scoring. A fine pass can keep one relevant sentence from an otherwise irrelevant paragraph, at the cost of much more work per request. Coarse mode can't slice into a paragraph; it keeps it or drops it.

query and target_compression_ratio work the same in both modes. Only the unit of decision changes.

When the default works

• Very long contexts. Well past the typical RAG chunk, token-level scoring's wall-clock cost grows quickly. That's a rough field observation, not a hard threshold.
• Structured documents where block boundaries already carry meaning. Markdown, knowledge-base articles, transcripts, code files. Keeping a whole markdown section is usually what you wanted anyway.
• Pre-filtering before a fine pass. Coarse-cut a 100K-token corpus down to 20K, then run a fine pass over the cut for precision.
• Real-time pipelines. Agent loops and chat backends where compression sits on the latency critical path and a few percent precision is an acceptable trade.

When to set coarse=false

• Short or medium inputs. The coarse savings don't matter at this scale, so take the precision.
• You need specific sentences from a mostly irrelevant paragraph. That's what token-level scoring is for.
• High-stakes RAG. A single relevant span shouldn't be dropped along with its paragraph.
• Output feeds a noise-sensitive downstream model. Tighter, sentence-level output leaves less for the next model to filter.

Example

You don't need to mention coarse at all; paragraph-level scoring is the default. The SDK client field defaults to None/undefined and is omitted from the wire. The backend then applies its own default of True, paragraph-level. Passing coarse=True explicitly is the same as omitting it.

To switch to token-level precision, set coarse=False (Python), coarse: false (TypeScript), or "coarse": false in the JSON body:

The tradeoff at a glance

Relationship to heuristic_chunking

heuristic_chunking is a separate boolean on the same request. It changes the chunker (paragraphs / code blocks / fixed-size), not the scoring granularity. The two knobs are independent: coarse picks the unit of decision, heuristic_chunking picks how the input is split before scoring.

Where coarse is accepted

You can pass coarse on compress, compress_stream, compress_batch, compress_async, and the middleware compression policy (compression={"coarse": ...}). It works on both latte_v1 and latte_v2; swap compression_model_name and the flag behaves the same: