Skip to content

generateStream: no empty-response guard, no fallback, and a raw provider error yielded as content #237

Description

@NathanTarbert

Split out of #231, which carries this as a bullet rather than a tracked item. #223's review said "tracking with #228" — that reference is wrong: #228 is the QA chat SSE cross-chunk buffer bug in apps/web/src/hooks/use-qa-chat.ts, a different subject. So this has had no issue of its own.

generate() was fixed by #223 (merged as 825d0981): it now reads every text block and throws 'Model response contained no usable text' on none, caught by the never-crash fallback. generateStream has the same two defects, both unfixed.

1. No empty-response guard, and no fallback at all

packages/outpost/ai/src/generator.ts:196-206 yields only content_block_delta/text_delta events. A completion with no text yields nothing — the consumer sees a successful empty stream. There is no degraded signal on this path and no apology fallback, so it is strictly worse than the pre-#223 non-streaming behaviour, which at least returned an empty string a caller could test.

2. The catch yields a raw provider error as ordinary content

:207-208:

} catch (error) {
    yield `\n\n_Error generating response: ${error instanceof Error ? error.message : String(error)}_`;
}

That is provider error text placed into the response body. pipeline.generateStreamingResponse buffers what it yields, so a 500 becomes publishable content. Trap rather than live bug — no production caller reaches it today, which is also why it has stayed unnoticed. Overlaps #146 (the streaming path still reads the pre-#143 groundedness signal) and should probably be fixed in the same pass.

Why it hasn't bitten yet

generateStream's only caller is pipeline.generateStreamingResponse, which nothing in the queue invokes. The moment streaming is wired to a real channel, both defects are live.

Test gaps

No coverage for an empty stream and none for the catch path. Both are straightforward with the aimock recipe #223 used — the empty case needs a mocked stream that yields no text delta.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: aiAI response pipeline, groundedness, confidenceroadmapTracked on the Outpost roadmaproadmap: nextRoadmap horizon: after launch path clears

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions