Skip to main content
Set "stream": true on POST /v1/chat/completions to receive the response as Server-Sent Events — the standard OpenAI chunk protocol with two additions worth knowing about.

​
The wire

​
Keep-alive pings

If the model is quiet for more than 8 seconds (long prompt prefill, deep reasoning), the stream emits an SSE comment line : ping so intermediate proxies don’t cut the idle connection. Standard SSE parsers ignore comment lines automatically; if you parse the stream by hand, skip lines starting with :.

​
The final frames

The last data frame before data: [DONE] carries usage and the wellspring quota receipt — the exact post-request figures (the response headers, which were flushed before generation, are only estimates on streams). See Quota feedback.

​
Reasoning deltas

On models with supports_reasoning, thinking text arrives as delta.reasoning — a separate channel from delta.content:
Clients that don’t care about reasoning can simply ignore the field. Reasoning always precedes content; once delta.content (or a tool call) starts, the reasoning phase is over.

​
Tool call deltas

Tool calls stream as incremental delta.tool_calls entries, OpenAI-style: index identifies the call being built, id/function.name arrive first, and function.arguments accumulates across chunks. When the model finishes with tool calls, finish_reason is tool_calls.

​
Disconnects and failures

  • A failure before any bytes were streamed returns a normal JSON error envelope — the request never becomes a stream.
  • A failure mid-stream ends the connection without data: [DONE]. Treat that as a failed request; do not trust partial output.
  • Keep your client’s idle timeout above ~10 seconds so keep-alive pings can reach you.