Skip to content

Feat: Add Batch API support - #401

Open
jsflax wants to merge 1 commit into
MacPaw:mainfrom
jsflax:feat/add-batch-api
Open

jsflax wants to merge 1 commit into
MacPaw:mainfrom
jsflax:feat/add-batch-api

Conversation

@jsflax

@jsflax jsflax commented Dec 26, 2025

Copy link
Copy Markdown

What

Add full support for OpenAI's Batch API, enabling asynchronous processing of large request volumes with 50% cost savings.

Core API methods:

  • createBatch - Create a batch from an uploaded JSONL file
  • retrieveBatch - Get batch status and details
  • listBatches - List batches with pagination
  • cancelBatch - Cancel an in-progress batch

File API additions:

  • retrieveFileContent - Download file content (needed for batch output)
  • deleteFile - Delete uploaded files

Convenience methods:

  • submitBatch - Handles JSONL encoding, file upload, and batch creation in one call
  • waitForBatch - Polls for completion and returns parsed responses

New types:

  • BatchQuery, BatchResult, BatchListResult, BatchResponseLine
  • BatchEndpoint, BatchCompletionWindow, BatchStatus enums
  • BatchError for convenience method error handling
  • FileDeleteResult for file deletion responses

Why

The Batch API offers significant benefits for processing large volumes of requests:

  • 50% cost reduction compared to synchronous API calls
  • Higher rate limits with a separate pool
  • 24-hour turnaround guarantee

This is particularly useful for bulk data processing, evaluations, benchmarks, and batch content generation.

Affected Areas

  • Sources/OpenAI/Public/Models/ - New Batch API models
  • Sources/OpenAI/Public/Protocols/ - Protocol extensions
  • Sources/OpenAI/OpenAI.swift - Closure-based implementations + API paths
  • Sources/OpenAI/OpenAI+OpenAIAsync.swift - Async implementations + convenience methods
  • Sources/OpenAI/Private/ - Request builders and raw data handling
  • Tests/OpenAITests/ - Unit and integration tests
  • README.md - Documentation

More Info

  • OpenAI Batch API Documentation
  • Includes both unit tests (BatchAPITests.swift) and integration tests (BatchAPIIntegrationTests.swift)
  • Integration tests verify structured output support with JSON schemas
  • All 5 integration tests passing against live API

Add full support for OpenAI's Batch API, which allows sending asynchronous
groups of requests with 50% lower costs and a 24-hour turnaround time.

Core API methods:
- createBatch: Create a batch from an uploaded JSONL file
- retrieveBatch: Get batch status and details
- listBatches: List batches with pagination
- cancelBatch: Cancel an in-progress batch

File API additions:
- retrieveFileContent: Download file content (for batch output)
- deleteFile: Delete uploaded files

Convenience methods:
- submitBatch: Handles JSONL encoding, file upload, and batch creation
- waitForBatch: Polls for completion and returns parsed responses

New types:
- BatchQuery, BatchResult, BatchListResult, BatchResponseLine
- BatchEndpoint, BatchCompletionWindow, BatchStatus enums
- BatchError for convenience method error handling
- FileDeleteResult for file deletion

Includes both unit tests (BatchAPITests) and integration tests
(BatchAPIIntegrationTests) with structured output support.

@Krivoblotsky Krivoblotsky left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this, and sorry it sat for so long. I checked it against current main: it merges cleanly, builds, and the whole test suite passes with it applied. Before it can land there are a few structural points, mostly about where the API surface goes rather than what it does.

1. Protocol requirements trip the new API gate. The repo now runs swift package diagnose-api-breaking-changes on every PR. This branch adds 12 requirements to public protocols (8 on OpenAIAsync, 4 on OpenAIProtocol), and every one is reported as a break because any external conformer (typically a test mock) stops compiling. The precedent we want to follow is the Responses API: one var responses: ResponsesEndpointProtocol { get } on OpenAIProtocol, and the methods live on the endpoint protocol with completion, async and Combine flavours. For this PR that means openAI.batches (BatchesEndpointProtocol with create, retrieve, list, cancel) and openAI.files for retrieveContent and delete. One new requirement per group instead of twelve, and the Combine flavour comes for free instead of being missing for the file methods. This is now the documented approach for every new endpoint group; the API stability section of CONTRIBUTING.md describes it once #440 lands.

2. The batch line types hardcode Chat Completions. BatchRequestLine.body is ChatQuery and BatchResponseBody.body is ChatResult, but the API accepts eight endpoints (/v1/responses, /v1/chat/completions, /v1/embeddings, /v1/completions, /v1/moderations, /v1/images/generations, /v1/images/edits, /v1/videos). The request line should be generic over an Encodable body, and the response line should expose the body as JSON with typed decoding helpers (decodeBody(as:)), so an embeddings or Responses batch works without new types.

3. BatchEndpoint and BatchCompletionWindow are closed enums. Adding a case later breaks exhaustive switch statements in client code, so under the additive-only policy they can never grow. Please make them RawRepresentable structs with static constants, like Model, and include the eight endpoints above.

4. Spec fields. BatchResult lacks errors, model and usage; BatchQuery lacks output_expires_after. errors matters most, it is the only way to see why a batch failed.

5. Convenience methods as requirements. submitBatch and waitForBatch are useful, but as protocol requirements they force every conformer to implement polling. Please move them to an extension with a default implementation on the endpoint protocol.

6. Tests. The integration tests skip without OPENAI_API_KEY, which is fine. The unit tests are great and mostly carry over.

maintainerCanModify is on, so if you would rather we do the restructuring on your branch, say so and we will, keeping your authorship. Either way this is the Batch API we want in the SDK; thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants