Conversation
7d5804b to
7d385dd
Compare
Krivoblotsky
left a comment
There was a problem hiding this comment.
Thank you for the work here, and apologies for the wait. It merges cleanly with main, builds, and all 217 tests pass. I want to be upfront about the direction, because it affects most of the diff.
ChatQuery already covers audio chat. On main today: ChatQuery.modalities, ChatQuery.audioOptions (11 voices including ash, ballad, coral, sage, verse; formats mp3, opus, flac, wav, pcm16), ContentPartAudioParam for input_audio, ChatResult.Choice.Message.audio (id, expiresAt, data, transcript) and the same on the streaming delta. AudioChatQuery, AudioChatResult and AudioChatStreamResult are a second way to call /chat/completions with a smaller surface (6 voices, fewer parameters), and audioChats / audioChatsStream add four protocol requirements, which the new API-breakage gate flags. The library should have one way to do a chat completion.
AudioConversationManager is conversation state management, which belongs in the app (or the Demo), not in the SDK.
What we would like to take from this PR:
- the
gpt-audio-1.5constant (plusgpt-audioandgpt-audio-mini), withgpt-4o-audio-previewmarked deprecated since it was shut down on 7 May 2026; - a README section "Audio input and output with Chat Completions" built on
ChatQuery, which the README lacks entirely today; your examples adapt almost one to one; - the coding tests, adapted to
ChatQuerywhere they add coverage we do not have.
If you are up for reshaping the PR along those lines, that would be great and it would land quickly. If not, we can open a smaller PR that carries those pieces with you credited as co-author. Either way, thank you for pushing on audio support.
Hey, thanks for the feedback, I'll reshape it in the next couple days |
- Add AudioChatQuery, AudioChatResult, AudioChatStreamResult models - Implement audioChats() and audioChatsStream() methods - Add type-safe enums: AudioFormat, Voice, Modality - Add AudioConversationManager for multi-turn conversations - Add support for gpt-4o-realtime-preview and gpt-4o-mini-realtime-preview models - Add dated variants: 2024-12-17 snapshots - Comprehensive test coverage (38 tests across 4 test suites) - Update README with audio chat documentation and examples - Format requirements: wav/mp3 for input, pcm16 recommended for streaming output - Relaxed parsing support for handling missing fields
58b84a0 to
a090543
Compare
Following review feedback on MacPaw#397, drop the parallel audio chat API and build on the ChatQuery surface the library already has. Removed: AudioChatQuery, AudioChatResult, AudioChatStreamResult, the audioChats/audioChatsStream protocol requirements, and AudioConversationManager (conversation state belongs in the app). Kept, per review: - gpt-audio-1.5, gpt-audio and gpt-audio-mini constants, with gpt-4o-audio-preview deprecated since it was shut down on 7 May 2026 and dropped from allModels(satisfying:), as other deprecated constants are - a README section, "Audio input and output with Chat Completions", built on ChatQuery modalities and audioOptions, covering audio in and out, text in and audio out, streaming, multi-turn via the audio id, and audio token usage - coding tests adapted to ChatQuery, covering input_audio content parts, assistant messages referencing a previous audio response, and decoding message and delta audio, none of which was covered before Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Following review feedback on MacPaw#397, drop the parallel audio chat API and build on the ChatQuery surface the library already has. Removed: AudioChatQuery, AudioChatResult, AudioChatStreamResult, the audioChats/audioChatsStream protocol requirements, and AudioConversationManager (conversation state belongs in the app). Kept, per review: - gpt-audio-1.5, gpt-audio and gpt-audio-mini constants, with gpt-4o-audio-preview deprecated since it was shut down on 7 May 2026 and dropped from allModels(satisfying:), as other deprecated constants are - a README section, "Audio input and output with Chat Completions", built on ChatQuery modalities and audioOptions, covering audio in and out, text in and audio out, streaming, multi-turn via the audio id, and audio token usage - coding tests adapted to ChatQuery, covering input_audio content parts, assistant messages referencing a previous audio response, and decoding message and delta audio, none of which was covered before
a090543 to
76af3ea
Compare
|
@Krivoblotsky done, let me know if I can help with anything else |
What
Adds audio support through the existing
ChatQuerysurface rather than a parallel API:gpt-audio-1.5,gpt-audioandgpt-audio-minimodel constantsgpt-4o-audio-previewmarked deprecated (shut down 7 May 2026) and dropped fromallModels(satisfying:), as other deprecated constants areChatQuery.modalitiesandChatQuery.audioOptions— audio in and out, text in with audio out, streaming, multi-turn via the audio id, and audio token usageinput_audiocontent parts, assistant messages referencing a previous audio response, and decodingmessage.audioand the streamingdelta.audioWhy
The first version of this PR added
AudioChatQuery,AudioChatResult,AudioChatStreamResult,audioChats/audioChatsStreamand anAudioConversationManager. Per review, that was a second way to call/chat/completionswith a smaller surface (6 voices against 11, fewer parameters), added four protocol requirements that the API-breakage gate flags, and put conversation state management inside the SDK. All of it is removed, so the library keeps one way to make a chat completion.What remains is what the library was actually missing: the current audio model constants, and documentation for audio over Chat Completions, which the README did not cover at all.
Affected Areas
Sources/OpenAI/Public/Models/Models/Models.swift— three new constants, one deprecationREADME.md— new section under AudioTests/OpenAITests/— audio cases added to the existingChatQuery,ChatResultandChatStreamResulttest filesNo API breakage:
swift package diagnose-api-breaking-changesagainst0.5.1reports none, so no allowlist entry is needed. The branch is rebased onmainand all 218 tests pass.