fix(provider): do not place Bedrock cachePoint after reasoning blocks - #358
Conversation
Bedrock rejects "ValidationException: Cache point cannot be inserted after reasoning block" when a cachePoint sits immediately after a reasoning block. Message-level bedrock.cachePoint was emitted after all content parts, so assistant turns that end in reasoning wedged the session on every retry (the turn fails, each retry rebuilds the same invalid request). Anchor the breakpoint on the last non-reasoning part (or skip entirely for reasoning-only messages), mirrored across the LLM cache policy and the Bedrock Converse lowering path. Cherry-picked from anomalyco/opencode#36532 (auto-closed unmerged by the PR-cleanup bot; never landed on the fork).
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Ports the six regression tests from anomalyco/opencode#36532: - cache-policy: anchors on text before trailing reasoning; skips reasoning-only messages - bedrock-converse: no cachePoint after a trailing reasoning block (#36517) - transform: bedrock cachePoint-after-reasoning describe (3 cases) All green; llm + opencode typecheck clean.
|
Superseded by harmoniqs/amicode#1209, which lands this exact fix in the correct place — the engine overlay ( |
Problem
On Amazon Bedrock (Claude with extended thinking), a prompt-cache breakpoint gets placed on the last couple of messages. The AI SDK emits message-level
bedrock.cachePointafter every content part, so when an assistant message ends in a reasoning block, Bedrock rejects the request:The turn fails, and every retry rebuilds the same invalid request, so the session is wedged until that message ages out of history. In Amicode this surfaces as the cryptic
undefined:prefix (the error is stringified as${type}: ${message}and Bedrock returns a nulltype). It hit a live session this week.Fix
Anchor the cache breakpoint on the last non-reasoning content part (or skip caching entirely for reasoning-only assistant messages), across both code paths that place the marker:
packages/llm/src/cache-policy.ts—markMessageAtprefers the last text/non-reasoning part.packages/opencode/src/provider/transform.ts—applyBedrockCacheOptionsmoves theproviderOptions.bedrock.cachePointoff a trailing reasoning block for Bedrock models.packages/llm/src/protocols/bedrock-converse.ts— comment marking the Converse lowering guard.Provenance
Faithful cherry-pick of
anomalyco/opencode#36532, which fixed upstream issue #36517. That PR was auto-closed unmerged by the cleanup bot (>1 month old, <2 reactions) — not rejected on merit — so it never landed here.Verification
e669df47b):cache-policy.test.ts— anchors on text before trailing reasoning; skips reasoning-only messagesbedrock-converse.test.ts— no cachePoint after a trailing reasoning block (#36517)transform.test.ts— bedrock cachePoint-after-reasoning describe (3 cases)bun test→ llm 43/43, opencode transform 402/402, 0 fail.bun typecheckpasses in bothpackages/llmandpackages/opencode.Note
A sibling variant — "There is nothing available to cache" (empty/below-min-size segment, open upstream #45855 / #45842) — is a separate bug, not covered here.
cc @jeonghun-jj-lee — this is the Bedrock reasoning-block cache crash; small, tested, green. Good to get into
local/amicode.