fix(mcp): make keyless recovery consent-first by hmishra2250 · Pull Request #363 · firecrawl/firecrawl-mcp-server

hmishra2250 · 2026-08-08T10:26:45Z

Why

Keyless recovery errors are read by an agent, but connection changes and credentials belong to a human or operator.

The old recovery text could make agents stop when Search, Scrape, or Parse could still finish the task. The longer replacement guidance fixed the safety problem, but it was too heavy for a simple account-only tool miss and caused agents to repeat the raw OAuth MCP endpoint.

What changed

Makes account-only tool recovery fallback-first: continue with Search, Scrape, or Parse when they can complete the task
Escalates to human setup only when the requested task truly needs the unavailable tool
Links account-only recovery to the neutral MCP setup guide instead of putting the OAuth MCP endpoint in model-visible prose
Keeps continue_keyless, human_reconnect_account, and operator_configure_api_key as structured actions
Keeps full human/operator guidance for quota exhaustion and invalid credentials
Keeps API keys outside agent chat and MCP URLs
Requires a new client session or run after a connection change
Returns structured invalid-OAuth recovery even when OAuth challenges are disabled
Adds direct tests for WWW-Authenticate escaping
Corrects the README keyless tool list and bumps the package to 3.23.7

Agent behavior experiment

Ran 52 valid neutral Claude Code and Codex CLI trials with a real local MCP fixture:

36 primary runs with Claude Opus and gpt-5.6-sol
16 lower-capability runs with Claude Haiku and gpt-5.6-terra
A fallback-friendly task with five known pricing URLs
A Crawl-required whole-site discovery task
No prompt instruction to block, avoid fallbacks, or avoid configuration changes

Results:

All fallback-friendly runs completed with keyless Scrape or Parse
The final short copy gave a safe human recovery path in all 10 Crawl-required trials
The final short copy exposed the raw OAuth MCP endpoint in 0 of 10 Crawl-required trials
No valid run tried to change MCP configuration
No valid run asked for a secret in chat
No valid run falsely claimed whole-site discovery was complete
Real Codex and Claude configuration files remained unchanged

The experiment measures how agents react after receiving the tool error. It does not measure natural first-tool selection. Managed Claude.ai and ChatGPT connectors were not testable from the local harness.

Validation

FIRECRAWL_MCP_SEARCH_PORT=39241 npm test (71 passing)
npx tsc --noEmit
npm run lint
npm pack --dry-run
git diff --check
Independent code review and verification found no blocking issues

Docs release order

The account-only message links to the neutral setup and recovery guide in firecrawl/firecrawl-docs#1234. That docs PR pins firecrawl-mcp 3.23.7, so it should merge immediately after this version is published.

Client boundary

Claude Code and Codex expose a startup 401 in client diagnostics, but do not currently pass it verbatim into the model context when MCP initialization fails. The HTTP recovery improves the human and diagnostic path. Connected tool-level failures remain the reliable agent-visible recovery path.

Review focus

Fallback-first behavior for account-only tools
Human/operator ownership of configuration and credentials
Structured recovery when OAuth challenges are disabled
The neutral docs handoff and release order

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

cubic-dev-ai

All reported issues were addressed across 8 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

… payload shape Addresses cubic P2 (src/index.ts:1007, confidence 6): the "never put an API key in chat/URL" and "retry after a new session/run" sentences were duplicated verbatim across HUMAN_CONNECTION_GUIDANCE, invalidApiKeyRecoveryPayload, and both invalidOAuthRecoveryPayload branches; the two invalidApiKeyRecoveryPayload/invalidOAuthRecoveryPayload builders also repeated the same code/auth_mode/docs_url/next_actions shape. Extracts NEVER_SHARE_API_KEY_SENTENCE and RETRY_AFTER_CONFIRM_SUFFIX as the single source for that phrasing, and a small connectionRecoveryPayload() base builder for the two recovery payloads. Pure dedup: every emitted message/payload byte is unchanged (verified by diffing the built dist output against the pre-refactor build). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

cubic-dev-ai

All reported issues were addressed across 1 file (changes from recent commits).

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

hmishra2250 · 2026-08-10T10:01:01Z

@cubic-dev-ai review

cubic-dev-ai · 2026-08-10T10:01:07Z

@cubic-dev-ai review

@hmishra2250 I have started the AI code review. It will take a few minutes to complete.

cubic-dev-ai

All reported issues were addressed across 8 files

_{Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic}

With CLOUD_SERVICE=false the invalid-OAuth recovery told the user to sign in through the client's account-connection flow even though this server cannot start one. The advertise branch now also requires isMcpOAuthEnabled(), matching the existing composition used for tool listing; disabled-OAuth deployments fall through to the guidance for servers that do not start account sign-in. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

_{Requires human review: Needs human judgment on the agent-facing credential-recovery policy and fallback-first behavior; the core recovery-payload logic is truncated, so the diff does not establish that this auth/credential-handling change is safe.

Re-trigger cubic}

erikengervall

lgtm with small nit

Pure relocation addressing review feedback: the guidance URL constants lived ~830 lines from the DEFAULT_* resource constants. They now share one section, with a comment explaining why MCP_OAUTH_SERVER_URL repeats the oauth resource value without aliasing it (protocol identity is env-overridable per deployment; the human-facing copy is fixed). No behavior change: bundle content is line-identical modulo ordering. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

cubic-dev-ai

0 issues found across 1 file (changes from recent commits).

_{Requires human review: Alters authentication/credential error handling and agent-visible recovery guidance (OAuth and API-key), a security and privacy policy change; a human should weigh fallback-first behavior and the new structured recovery payloads.

Re-trigger cubic}

* fix(mcp): make invalid API key recovery reach the agent An invalid or revoked API key on /v2/mcp returned HTTP 401 at initialize and tools/list. MCP clients treat a 401 at connect as "server unavailable" and never surface the response body to the model, so the CREDENTIAL_INVALID recovery payload — the whole point of #363 for this case — was unreachable in a real agent session. The agent saw "no Firecrawl tools connected" and fell back to generic, often outdated setup advice. Fix: on the keyless+API-key endpoint, admit a supplied-but-invalid credential as a session flagged credentialError instead of throwing a 401. The connection succeeds, tools list, and every tool call returns the CREDENTIAL_INVALID recovery payload as a 200 isError result — the same agent-legible path keyless quota recovery already uses. No credential is forwarded and no tool executes, so this grants zero functional access (strictly less than the previously rejected "downgrade to keyless" option). Scope: only profiles that accept API keys as a valid auth mode (allowKeyless). OAuth-only surfaces such as /v2/mcp-search keep their hard 401 credential rejection unchanged. Verified end to end: a live claude session wired to an invalid-key server now receives the payload and relays both recovery options to the human without asking for a key in chat. Smoke suite updated to the 200/isError contract; no net-new failures vs main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(mcp): bump to 3.23.8 for the invalid-key recovery fix Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(mcp): stop CREDENTIAL_INVALID payload from advertising unusable keyless tools The credentialError guard in guardHostedTool's execute() runs before the keyless branch, so no tool (including keyless ones) is actually callable in a CREDENTIAL_INVALID session. recoveryPayload() was still including available_tools: KEYLESS_TOOL_NAMES for this code, which could send the agent into a retry loop against tools that just return the same recovery error. Omit available_tools when code is CREDENTIAL_INVALID, matching the existing isKeylessAccessUnavailable carve-out. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(mcp): scope CREDENTIAL_INVALID tools/list to the keyless surface; pin next_actions shape canList previously returned true for every registered tool in a credentialError session, so tools/list disclosed the full authenticated tool schema (including non-keyless tools) to any request carrying an unrecognized or invalid credential -- more schema disclosure than a real keyless session gets. Scope it to the same keyless-tool surface a keyless session already lists; recovery guidance still surfaces on any listed tool call since execute() gates on credentialError before the keyless branch either way. Also tighten the invalid-key next_actions assertion in mcp-smoke.test.mjs from a loose non-empty-array check to pinning the human_reconnect_account / operator_configure_api_key kinds and their consent flags, so a regression that drops or reorders the recovery actions is caught. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

This was referenced Aug 8, 2026

docs(mcp): route setup by user outcome firecrawl/firecrawl-docs#1234

Merged

fix(mcp): stop keyless setup from triggering OAuth #364

Merged

hmishra2250 added 3 commits August 9, 2026 02:12

fix(mcp): make keyless recovery consent-first

919cbe6

fix(mcp): complete safe keyless recovery

ddaf623

fix(mcp): guide invalid credential recovery

cf1030e

hmishra2250 force-pushed the fix/keyless-recovery-consent-guidance branch from d9d8be9 to cf1030e Compare August 8, 2026 20:52

fix(mcp): make account-only recovery fallback-first

2ba6156

hmishra2250 marked this pull request as ready for review August 8, 2026 22:48

cubic-dev-ai Bot reviewed Aug 8, 2026

View reviewed changes

Comment thread src/index.ts

cubic-dev-ai Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread src/index.ts

cubic-dev-ai Bot reviewed Aug 10, 2026

View reviewed changes

Comment thread src/index.ts Outdated

cubic-dev-ai Bot reviewed Aug 10, 2026

View reviewed changes

erikengervall reviewed Aug 10, 2026

View reviewed changes

Comment thread src/index.ts Outdated

erikengervall approved these changes Aug 10, 2026

View reviewed changes

cubic-dev-ai Bot reviewed Aug 10, 2026

View reviewed changes

hmishra2250 merged commit 2eb135a into main Aug 10, 2026
2 checks passed

hmishra2250 mentioned this pull request Aug 10, 2026

fix(mcp): make invalid API key recovery reach the agent #365

Merged

Navigation Menu

Uh oh!

fix(mcp): make keyless recovery consent-first - #363

fix(mcp): make keyless recovery consent-first#363
hmishra2250 merged 7 commits into
mainfrom
fix/keyless-recovery-consent-guidance

hmishra2250 commented Aug 8, 2026 •

edited

Loading

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

hmishra2250 commented Aug 10, 2026

Uh oh!

cubic-dev-ai Bot commented Aug 10, 2026

Uh oh!

cubic-dev-ai Bot left a comment •

edited

Loading

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Uh oh!

erikengervall left a comment

Uh oh!

cubic-dev-ai Bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

hmishra2250 commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What changed

Agent behavior experiment

Validation

Docs release order

Client boundary

Review focus

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hmishra2250 commented Aug 10, 2026

Uh oh!

cubic-dev-ai Bot commented Aug 10, 2026

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erikengervall left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

hmishra2250 commented Aug 8, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading