Skip to main content

stream_response

flow_adapter_ai::providers::openai_compat

Function stream_response 

Source 
pub async fn stream_response(
    builder: RequestBuilder,
    provider: &str,
    call_id: &str,
    sink: &dyn LlmStreamSink,
) -> Result<CloudAiResponse, CloudAiError>
Expand description

Stream an OpenAI-compatible Chat Completions response, emitting each content delta through sink. Returns the accumulated full response once the stream closes so the caller still gets a CloudAiResponse to surface in the node output.

Handles the canonical SSE shape (one data: <json>\n per chunk plus a terminating data: [DONE]). Servers that don’t include a usage chunk leave token counts at zero, mirroring the non-streaming path.