Skip to content

fix(chat): handle out-of-order streamed choice indexes - #3709

Open
HostX0 wants to merge 1 commit into
openai:mainfrom
HostX0:fix/chat-stream-choice-index-order
Open

fix(chat): handle out-of-order streamed choice indexes#3709
HostX0 wants to merge 1 commit into
openai:mainfrom
HostX0:fix/chat-stream-choice-index-order

Conversation

@HostX0

@HostX0 HostX0 commented Aug 21, 2026

Copy link
Copy Markdown
  • I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested

Fix Chat Completions stream accumulation when choice chunks arrive out of index order.

ChatCompletionStreamState currently uses choice.index as a Python list offset in the snapshot and event-state collections. That only works when every lower-numbered choice has already appeared. A chunk containing a single choice with index=1, for example, has a one-element choices list, so the initial snapshot path attempts to assign choices[1] and raises IndexError.

This change keeps the API-provided index as the choice identity instead of assuming it is the current list position:

  • key per-choice event state by choice.index;
  • resolve accumulated snapshots by their stored index field;
  • keep newly discovered snapshots ordered by index for the final completion;
  • transform the choices present in the initial chunk by their actual list positions rather than their logical indexes.

The regression test feeds the accumulator a valid 1 -> 0 -> 1 sequence and verifies that the later delta is appended to choice 1, while the final snapshot remains ordered as choices 0 and 1.

Additional context & links

The production change is confined to the handwritten src/openai/lib/streaming/chat/ implementation; no generated API resource or public model changes are required.

@HostX0
HostX0 requested a review from a team as a code owner August 21, 2026 04:43
@HostX0
HostX0 force-pushed the fix/chat-stream-choice-index-order branch from ad1e2a7 to 1440abb Compare August 21, 2026 04:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant