Skip to content

OpenAiSDK Stream does not have real-time output #5162

@ChasionNo1

Description

@ChasionNo1

Please do a quick search on GitHub issues first, there might be already a duplicate issue for the one you are about to create.
If the bug is trivial, just go ahead and create the issue. Otherwise, please take a few moments and fill in the following sections:

Bug description
A clear and concise description of what the bug is about.
When I use OpenAiSDK's stream, the output results are not output in real-time, and I wait for all results to be ready before outputting all streams at once.
Environment
Please provide as many details as possible: Spring AI version, Java version, which vector store you use if any, etc
1.1.2
Steps to reproduce
Steps to reproduce the issue.

Expected behavior
A clear and concise description of what you expected to happen.

Minimal Complete Reproducible example
Please provide a failing test or a minimal complete verifiable example that reproduces the issue.
Bug reports that are reproducible will take priority in resolution over reports that are not reproducible.

I am using the official provided example, as follows:

@GetMapping(value = "/ai/generateStream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<ChatResponse> generateStream(
        @RequestParam(value = "message", defaultValue = "Tell me a joke") String message) {
    Prompt prompt = new Prompt(new UserMessage(message));
    return chatModel.stream(prompt);
}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions