Conversation
Add full support for OpenAI's Batch API, which allows sending asynchronous groups of requests with 50% lower costs and a 24-hour turnaround time. Core API methods: - createBatch: Create a batch from an uploaded JSONL file - retrieveBatch: Get batch status and details - listBatches: List batches with pagination - cancelBatch: Cancel an in-progress batch File API additions: - retrieveFileContent: Download file content (for batch output) - deleteFile: Delete uploaded files Convenience methods: - submitBatch: Handles JSONL encoding, file upload, and batch creation - waitForBatch: Polls for completion and returns parsed responses New types: - BatchQuery, BatchResult, BatchListResult, BatchResponseLine - BatchEndpoint, BatchCompletionWindow, BatchStatus enums - BatchError for convenience method error handling - FileDeleteResult for file deletion Includes both unit tests (BatchAPITests) and integration tests (BatchAPIIntegrationTests) with structured output support.
Krivoblotsky
left a comment
There was a problem hiding this comment.
Thank you for this, and sorry it sat for so long. I checked it against current main: it merges cleanly, builds, and the whole test suite passes with it applied. Before it can land there are a few structural points, mostly about where the API surface goes rather than what it does.
1. Protocol requirements trip the new API gate. The repo now runs swift package diagnose-api-breaking-changes on every PR. This branch adds 12 requirements to public protocols (8 on OpenAIAsync, 4 on OpenAIProtocol), and every one is reported as a break because any external conformer (typically a test mock) stops compiling. The precedent we want to follow is the Responses API: one var responses: ResponsesEndpointProtocol { get } on OpenAIProtocol, and the methods live on the endpoint protocol with completion, async and Combine flavours. For this PR that means openAI.batches (BatchesEndpointProtocol with create, retrieve, list, cancel) and openAI.files for retrieveContent and delete. One new requirement per group instead of twelve, and the Combine flavour comes for free instead of being missing for the file methods. This is now the documented approach for every new endpoint group; the API stability section of CONTRIBUTING.md describes it once #440 lands.
2. The batch line types hardcode Chat Completions. BatchRequestLine.body is ChatQuery and BatchResponseBody.body is ChatResult, but the API accepts eight endpoints (/v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations, /v1/images/generations, /v1/images/edits, /v1/videos). The request line should be generic over an Encodable body, and the response line should expose the body as JSON with typed decoding helpers (decodeBody(as:)), so an embeddings or Responses batch works without new types.
3. BatchEndpoint and BatchCompletionWindow are closed enums. Adding a case later breaks exhaustive switch statements in client code, so under the additive-only policy they can never grow. Please make them RawRepresentable structs with static constants, like Model, and include the eight endpoints above.
4. Spec fields. BatchResult lacks errors, model and usage; BatchQuery lacks output_expires_after. errors matters most, it is the only way to see why a batch failed.
5. Convenience methods as requirements. submitBatch and waitForBatch are useful, but as protocol requirements they force every conformer to implement polling. Please move them to an extension with a default implementation on the endpoint protocol.
6. Tests. The integration tests skip without OPENAI_API_KEY, which is fine. The unit tests are great and mostly carry over.
maintainerCanModify is on, so if you would rather we do the restructuring on your branch, say so and we will, keeping your authorship. Either way this is the Batch API we want in the SDK; thanks again.
What
Add full support for OpenAI's Batch API, enabling asynchronous processing of large request volumes with 50% cost savings.
Core API methods:
createBatch- Create a batch from an uploaded JSONL fileretrieveBatch- Get batch status and detailslistBatches- List batches with paginationcancelBatch- Cancel an in-progress batchFile API additions:
retrieveFileContent- Download file content (needed for batch output)deleteFile- Delete uploaded filesConvenience methods:
submitBatch- Handles JSONL encoding, file upload, and batch creation in one callwaitForBatch- Polls for completion and returns parsed responsesNew types:
BatchQuery,BatchResult,BatchListResult,BatchResponseLineBatchEndpoint,BatchCompletionWindow,BatchStatusenumsBatchErrorfor convenience method error handlingFileDeleteResultfor file deletion responsesWhy
The Batch API offers significant benefits for processing large volumes of requests:
This is particularly useful for bulk data processing, evaluations, benchmarks, and batch content generation.
Affected Areas
Sources/OpenAI/Public/Models/- New Batch API modelsSources/OpenAI/Public/Protocols/- Protocol extensionsSources/OpenAI/OpenAI.swift- Closure-based implementations + API pathsSources/OpenAI/OpenAI+OpenAIAsync.swift- Async implementations + convenience methodsSources/OpenAI/Private/- Request builders and raw data handlingTests/OpenAITests/- Unit and integration testsREADME.md- DocumentationMore Info
BatchAPITests.swift) and integration tests (BatchAPIIntegrationTests.swift)