Skip to content

fix(provider): do not place Bedrock cachePoint after reasoning blocks - #358

Merged
jacktrnr merged 2 commits into
local/amicodefrom
bedrock-cache-reasoning
Sep 15, 2026
Merged

jacktrnr merged 2 commits into
local/amicodefrom
bedrock-cache-reasoning

Conversation

@jacktrnr

@jacktrnr jacktrnr commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Problem

On Amazon Bedrock (Claude with extended thinking), a prompt-cache breakpoint gets placed on the last couple of messages. The AI SDK emits message-level bedrock.cachePoint after every content part, so when an assistant message ends in a reasoning block, Bedrock rejects the request:

ValidationException: Cache point cannot be inserted after reasoning block.

The turn fails, and every retry rebuilds the same invalid request, so the session is wedged until that message ages out of history. In Amicode this surfaces as the cryptic undefined: prefix (the error is stringified as ${type}: ${message} and Bedrock returns a null type). It hit a live session this week.

Fix

Anchor the cache breakpoint on the last non-reasoning content part (or skip caching entirely for reasoning-only assistant messages), across both code paths that place the marker:

  • packages/llm/src/cache-policy.tsmarkMessageAt prefers the last text/non-reasoning part.
  • packages/opencode/src/provider/transform.tsapplyBedrockCacheOptions moves the providerOptions.bedrock.cachePoint off a trailing reasoning block for Bedrock models.
  • packages/llm/src/protocols/bedrock-converse.ts — comment marking the Converse lowering guard.

Provenance

Faithful cherry-pick of anomalyco/opencode#36532, which fixed upstream issue #36517. That PR was auto-closed unmerged by the cleanup bot (>1 month old, <2 reactions) — not rejected on merit — so it never landed here.

Verification

  • 6 regression tests ported from #36532 and all green (commit e669df47b):
    • cache-policy.test.ts — anchors on text before trailing reasoning; skips reasoning-only messages
    • bedrock-converse.test.ts — no cachePoint after a trailing reasoning block (#36517)
    • transform.test.ts — bedrock cachePoint-after-reasoning describe (3 cases)
  • Full suites pass: bun testllm 43/43, opencode transform 402/402, 0 fail.
  • bun typecheck passes in both packages/llm and packages/opencode.

Note

A sibling variant — "There is nothing available to cache" (empty/below-min-size segment, open upstream #45855 / #45842) — is a separate bug, not covered here.

cc @jeonghun-jj-lee — this is the Bedrock reasoning-block cache crash; small, tested, green. Good to get into local/amicode.

Bedrock rejects "ValidationException: Cache point cannot be inserted
after reasoning block" when a cachePoint sits immediately after a
reasoning block. Message-level bedrock.cachePoint was emitted after all
content parts, so assistant turns that end in reasoning wedged the
session on every retry (the turn fails, each retry rebuilds the same
invalid request).

Anchor the breakpoint on the last non-reasoning part (or skip entirely
for reasoning-only messages), mirrored across the LLM cache policy and
the Bedrock Converse lowering path.

Cherry-picked from anomalyco/opencode#36532 (auto-closed unmerged by the
PR-cleanup bot; never landed on the fork).
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 5cf05919-9873-4a76-ad74-8fdef5b97a31


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Ports the six regression tests from anomalyco/opencode#36532:
- cache-policy: anchors on text before trailing reasoning; skips
  reasoning-only messages
- bedrock-converse: no cachePoint after a trailing reasoning block (#36517)
- transform: bedrock cachePoint-after-reasoning describe (3 cases)

All green; llm + opencode typecheck clean.
@jacktrnr
jacktrnr merged commit 4e72a5f into local/amicode Sep 15, 2026
4 checks passed
@jacktrnr

Copy link
Copy Markdown
Contributor Author

Superseded by harmoniqs/amicode#1209, which lands this exact fix in the correct place — the engine overlay (packages/app-bundle/overlay/) that the Amicode binary is actually built from. This repo/fork is no longer the build source ("the fork is now archived; all engine development happens directly in the overlay"), so this merge is inert for the product. Kept for history; no action needed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant