Skip to content

ci: review MCP contract changes for docs impact - #349

Draft
DanielleWashington wants to merge 1 commit into
firecrawl:mainfrom
DanielleWashington:ci/docs-impact-review
Draft

ci: review MCP contract changes for docs impact#349
DanielleWashington wants to merge 1 commit into
firecrawl:mainfrom
DanielleWashington:ci/docs-impact-review

Conversation

@DanielleWashington

@DanielleWashington DanielleWashington commented Aug 3, 2026

Copy link
Copy Markdown

Why

firecrawl-docs#1195 establishes the deterministic reconciliation baseline and initial drift report. This linked follow-up adds advisory semantic review for MCP contract changes that fixed extractors may not cover.

Summary

  • Run an advisory Codex review when a relevant internal MCP pull request is opened, updated, reopened, or marked ready for review.
  • Inspect tool names, schemas, required fields, defaults, annotations, serialization, authentication, keyless behavior, transports, installation, and distribution.
  • Compare the diff with current firecrawl-docs and return No impact, Docs gap, or Ambiguous.
  • Upsert one PR comment with evidence, affected docs, the smallest next action, and an optional proposed docs patch.
  • Keep analysis read-only and isolate comment permissions in a separate job. The agent cannot edit files, push branches, open PRs, or merge changes.

Safety and activation gates

  • Depends on firecrawl-docs#1195 so the agent reads the merged reconciliation baseline from docs main.
  • Requires a repository or organization OPENAI_API_KEY; without it, the job emits a notice and exits successfully.
  • Runs only for non-draft, same-repository PRs. Fork PR coverage is intentionally deferred because Actions secrets are not exposed to untrusted forks.
  • The review is advisory and is not a required check.

Test plan

  • Parse the workflow as YAML.
  • Syntax-check the embedded GitHub Script.
  • Verify immutable action pins, path coverage, per-PR cancellation, fork/draft guards, read-only sandboxing, and isolated write permissions.
  • Run git diff --check and confirm only the workflow and prompt are added.
  • After the API key is configured, verify an internal MCP contract change receives one updated documentation-impact comment on successive pushes.

No MCP runtime behavior or documentation content changes in this PR.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Adds a read-only CI check that reviews MCP contract diffs for documentation impact and comments the outcome on PRs. It compares the PR changes against firecrawl-docs main and reports “No impact”, “Docs gap”, or “Ambiguous”.

  • New Features

    • Adds .github/workflows/docs-impact.yml and .github/prompts/docs-impact.md using openai/codex-action.
    • Runs on internal, non-draft PRs touching MCP-facing files; reads _docs/, classifies impact, and upserts a single PR comment with evidence and a smallest next step.
    • Sandboxed analysis with split comment job; advisory only (not a required check).
  • Migration

    • Add OPENAI_API_KEY as a repo or org secret to enable the check.
    • Ensure the docs reconciliation baseline exists on firecrawl-docs main.

Written for commit 42d65ae. Summary will update on new commits.

Review in cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant