Guides

Streaming compression

How the SSE streaming endpoint behaves today, how the SDK iterators consume it, and when to use plain compress instead.

Use compress_stream / compressStream to consume /api/compress/question-specific/stream as an iterator. The endpoint returns Server-Sent Events: data: frames only, no event:, id:, or retry: lines.

What streaming does today: the server compresses the full input, then sends the complete output as one final event. There is no token-by-token delivery yet, so streaming has no time-to-first-token advantage over the plain compress call. The iterator interface exists for forward-compatibility: if incremental delivery ships, code written against it won't need to change.

The wire protocol

On success the server sends one content event, then closes the connection:

text

If compression fails or exceeds the 120-second server-side budget after the stream opens, the server sends an error event, then closes:

text

Failures before the stream opens (bad key, validation, rate limit) arrive as normal HTTP error responses with the standard error envelope.

Consuming the stream

Iterate, concatenate content across chunks, and stop on done. Both SDKs yield each content event as {content, done: false}, then a final synthetic {content: "", done: true} when the stream closes. That pattern keeps working whether the output arrives in one event (today) or many (future).

python

The stream carries text only. For the token-accounting metadata (original_tokens, compressed_tokens, tokens_saved, actual_compression_ratio, duration_ms), use the non-streaming compress endpoint.

Handling errors

Handle errors by when they happen:

Before the stream opens. The SDK's initial status check maps 4xx / 5xx responses to typed errors (RateLimitError, AuthenticationError, CompresrError, ...). Catch these to back off or re-auth.
After the stream opens. A server-side failure arrives as an error event on the wire. The current SDK iterators skip events without a content field. So a mid-stream server error ends the iteration with whatever content arrived, possibly none, instead of raising. If an empty result is unacceptable, treat an empty buffer as a failure and retry. Transport-level drops do raise: httpx.HTTPError in Python, ConnectionError in TypeScript.
Retries. RetryConfig applies only to the initial connect, never to a mid-stream failure. retry_after / retryAfter is nullable; guard the sleep.

python

SDK caveats

Python is sync-only. compress_stream returns a Generator, not an AsyncGenerator. There is no compress_stream_async. On FastAPI or asyncio, offload the loop with asyncio.to_thread, or use the TypeScript SDK, which is async-native.
The TypeScript timeout covers the whole stream. The client arms one AbortController for timeout ms from connect. It is not idle-based. Long compressions may need a larger timeout at client construction.
Agent-layer streaming is not wired up. client.messages.stream(...) and client.chat.completions.stream(...) on the TS agent interfaces throw CompresrError('streaming not yet implemented', code: 'not_implemented'). The Python interfaces don't expose .stream(...) at all. compress_stream / compressStream is the working streaming path.

When to use it

Use the plain compress call for almost everything today. It returns the same output plus the token-accounting metadata, and it works fully with typed errors and RetryConfig. Use the streaming endpoint only if you're deliberately writing against the iterator interface for forward-compatibility. Use compress_batch when you have many contexts to send at once.