Skip to content

feat: pre-serve static-analysis gate for preview deploys - #34

Merged
luthermonson merged 4 commits into
mainfrom
feat/analyze-gate
Sep 13, 2026
Merged

luthermonson merged 4 commits into
mainfrom
feat/analyze-gate

Conversation

@luthermonson

Copy link
Copy Markdown
Contributor

What

Adds an opt-in pre-serve static-analysis gate to the preview-deployment daemon. After a PR's checkout is materialized and before the atomic swap makes the vhost routable — and before build:/seed: execute any of it — switchboard runs:

ephpm analyze <checkout> --config <operator-policy.yml> --format sarif

and refuses to publish on a bad verdict. On a redeploy the previous known-good container stays live (the swap simply never happens).

Insertion point

deployer::deploy_preview, new step (2b) — right after the framework/manifest load + validate_docroot, before materialize_env (step 3). This is deliberately before any external provisioning:

  • before env injection, so the analyzer never scans switchboard's own resolved secrets (which secrets-scan would otherwise flag every gated deploy);
  • before build:/seed:, so a composer-scripts finding blocks the code before its scripts run;
  • before the atomic swap (the true go-live), so a block means the site is never served and a block needs no teardown — only the staging tree is removed.

Behavior

  • Disabled unless --analyze-config is set (safe rollout default). Startup logs one WARN when off. The gate is live only when the config is set and --ephpm-bin supports analyze.
  • --config is always explicit, overriding ephpm analyze's auto-discovery of a .ephpm-analyze.yml inside the checkout — a malicious PR can't ship enable: [] to neuter its own gate. The analyzer is never run with the checkout as cwd.
  • Fail closed. Exit 0 publishes; 2 (quarantine) and 3 (deny) block; 1 (analyzer error), any other code, a timeout (--analyze-timeout-secs, default 180s), and a spawn failure all block.
  • Blocked previews are reported. The verdict, finding count, and top ~10 findings (rule / file:line / message) are folded into the PR's sticky comment; the GitHub deployment status is set to failure; the job lands in claimed/ for inspection.

Config / docs

  • New flags: --analyze-config (SWITCHBOARD_ANALYZE_CONFIG), --analyze-timeout-secs (SWITCHBOARD_ANALYZE_TIMEOUT_SECS, default 180). Reuses --ephpm-bin.
  • docs/analyze-gate.example.yml: reviewed native-only policy (six fast analyzers). opcode-scan is documented as an opt-in per-node upgrade (compiles every file via Zend, +5–15s on a full WP tree). wp-vuln stays out of required (optional feed).
  • README: new narrative section, config table, source-map row.

Tests

  • analyze.rs: pure decide() exhaustively (0→proceed, 1/2/3/other→block, timeout→block, unrunnable→block); --analyze-config: NoneSkipped without touching the binary; SARIF parsing incl. missing fields, cap-with-honest-total, unparseable input.
  • config.rs: gate off by default, flags parse, zero-timeout floored.
  • github.rs: blocked comment renders verdict + findings, caps at 10 with "Showing N of M", no-findings path, table-cell sanitization.

cargo fmt --all -- --check, cargo clippy --all-targets -- -D warnings, and cargo test (288 pass) all green.

Do not merge / do not deploy — this is outward-facing deploy infra for review first. The infra-side flag + config file are a separate follow-up in switchboard-infra.

Add an opt-in gate that runs `ephpm analyze <checkout> --config <operator
policy> --format sarif` over a PR's materialized checkout after it is fetched
and BEFORE the atomic swap makes the vhost routable (and before build:/seed:
execute any of it). A bad verdict blocks the preview.

- New src/analyze.rs: pure exit-code -> proceed/block decision (0 proceed;
  2 quarantine / 3 deny block; 1 analyzer-error, any other code, timeout, and
  a spawn failure all fail closed), lenient SARIF finding parsing, and the
  async runner (kill-on-drop, wall-clock timeout).
- Gate is DISABLED unless --analyze-config is set (safe rollout default);
  startup logs one WARN when off. --config is always explicit so a PR's own
  .ephpm-analyze.yml cannot neuter it; the analyzer is never run cwd=checkout.
- Screens the pristine code (before env injection) so it never scans
  switchboard's own resolved secrets.
- Blocked previews are reported on the PR's sticky comment (verdict, finding
  count, top ~10 findings as rule/file:line/message) and the deployment status
  is set to failure; the job lands in claimed/ for inspection. A block rolls
  back only the staging tree (nothing external was provisioned yet).
- Config: --analyze-config, --analyze-timeout-secs (default 180s); reuses
  --ephpm-bin. docs/analyze-gate.example.yml ships a reviewed native-only
  policy (opcode-scan documented as an opt-in per-node upgrade). README updated.
@ephpm

ephpm Bot commented Sep 13, 2026

Copy link
Copy Markdown

ePHPm Preview — removed

Preview deployment has been torn down.

@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 16:28 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 16:28 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 16:28 Inactive
The gate verdict is a pure function of (repo, PR head SHA, gate config), so it
is identical on every node. switchboard runs on every node, so a naive gate
re-scans the same commit N times. Compute it once and share it through ePHPm's
gossip-replicated KV.

- New kv::VerdictCache (GET + SET EX) over the same per-site AUTH scoping the
  share-link revoker uses; RESP bulk-reply reader added.
- analyze: CachedVerdict (serialize/deserialize verdict + findings), pure
  plan_from_lookup (hit->reuse, unparseable/miss/unavailable->scan), verdict_key
  (analyze:verdict:<repo>:<pr>:<sha>:<cfg_hash>), config_fingerprint, and
  run_gate_cached wrapping run_gate.
- Fail-closed on the gate, fail-SAFE on the dedup: KV read error -> scan
  locally; KV write error -> proceed with local verdict. Coordination failure
  degrades to per-node scanning, never to serve-unscanned. No lock/lease/leader
  wait (benign 2x race). Cache active only when --analyze-config AND
  --kv-secret-file are set; verdict stored in the preview's own keyspace (can
  only reinforce a decision, never escalate).
- Cache-hit reconstructs the identical block comment from the stored
  AnalyzeBlock (verdict + findings) via CachedVerdict::into_result.
- Config: --analyze-verdict-ttl-secs (default 1d); --kv-secret-file now dual-use.
  main logs dedup status once at startup. README documents once-per-commit
  cluster-wide scanning, fail-safe to per-node.
@luthermonson

Copy link
Copy Markdown
Contributor Author

Follow-up commit d77165c: dedup the verdict once per cluster instead of once per node.

Channel: ePHPm's per-site KV is gossip-replicated across the cluster (documented in kv.rs and reused by the existing share-link revoker), so it's a genuine cluster-shared store — no GitHub-polling hack needed. Reached over the RESP listener (--kv-addr) with the per-site AUTH <site> <HMAC(kv_secret,site)> scoping, available when --kv-secret-file is set.

Key: analyze:verdict:<repo>:<pr>:<head_sha>:<cfg_hash> where cfg_hash = SHA-256 fingerprint of the operator policy file contents (a policy edit re-scans everywhere). head_sha comes from the job (PreviewRequest::sha). Stored in the preview's own per-site keyspace with TTL --analyze-verdict-ttl-secs (default 1d; the SHA is the real invalidator).

Flow: cache hit → reuse the peer verdict, no scan; miss/unparseable/unavailable → scan then publish.

Fail-closed gate, fail-SAFE dedup: KV read error → scan locally; KV write error → proceed with the local verdict + warn. Coordination failure degrades to today's per-node scanning, never to serve-unscanned. No lock/lease/leader wait — a simultaneous double-miss just scans twice (identical result).

Cache-hit reconstructs the block comment by serializing the whole AnalyzeBlock (verdict + reason + findings + total) as CachedVerdict; a peer deserializes and into_result()s it, so format_block_comment renders byte-identically without re-scanning.

Security note: the verdict lives in the (untrusted) tenant's own keyspace, but it can't escalate — the first proceed for a SHA is always a real scan (the app can't run until a node has served it, which requires that node to have proceeded on its own scan), and the key is bound to the exact SHA + config fingerprint, so the app can't forge a future push's key.

Tests: pure plan_from_lookup (hit→reuse, miss/unavailable/unparseable→scan), CachedVerdict round-trip, verdict_key/config_fingerprint, and VerdictCache GET-hit/GET-miss/SET-EX/dead-addr against a fake RESP server. 304 tests green; fmt + clippy -D warnings clean. Still do not merge / do not deploy.

@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:04 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:04 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:04 Inactive
…espace

The verdict cache stored under the preview's own per-site KV keyspace was
poisonable: the running (untrusted) preview app can ephpm_kv_set a forged
Passed verdict for a FUTURE commit it authors — it knows repo/pr/SHA and the
cfg_hash is a truncated SHA-256 of the operator policy, which is effectively
public — so a peer would GET the forged pass, skip the scan, and serve the
payload. The prior "can't forge a future key" reasoning was wrong.

Fix: store verdicts under a reserved AUTH site VERDICT_STORE_SITE
("\x1fswitchboard-verdicts") that no tenant can reach. Disjointness: a
tenant's ephpm_kv_* is auto-scoped server-side by ephpm to its own resolved
site key, which is always a VALID site key ([a-z0-9._-]); the reserved
namespace leads with the gossip unit-separator \x1f, outside that charset, so
it can never equal any preview site key. switchboard holds kv_secret and can
AUTH as any site, so it alone addresses the namespace. All nodes AUTH the same
reserved site, so cross-node dedup is unaffected; the per-commit key keeps
distinct previews from colliding within the one namespace.

- VerdictCache drops its preview-site param; AUTHs as VERDICT_STORE_SITE.
- New test verdict_namespace_is_unreachable_by_any_tenant (reserved site is
  not a valid site key; disjoint from real site keys) and
  all_previews_share_one_verdict_namespace (fleet-wide dedup preserved).
- kv.rs security paragraph + README corrected; dropped the wrong
  can't-forge-a-future-key claim.
@luthermonson

Copy link
Copy Markdown
Contributor Author

⚠️ Security correction (commit 0b7f0a7). My earlier note claimed the verdict cache in the preview's own keyspace was safe because the app "can't forge a future push's key." That was wrong and I've fixed it.

The hole: the running preview app can ephpm_kv_set a forged Passed verdict for a future malicious commit it authors — it knows repo/PR/SHA_B, and cfg_hash is a truncated SHA-256 of the operator policy file, which is effectively public (operators copy the shipped example). A peer would then GET the forged pass, skip the scan, and serve the payload. Gate bypassed.

The fix: verdicts now live in a switchboard-private KV namespace, VERDICT_STORE_SITE = "\x1fswitchboard-verdicts", that no preview tenant can authenticate to.

Disjointness argument: a tenant's ephpm_kv_* is auto-scoped server-side by ePHPm to the request's own resolved site key — a tenant never chooses an AUTH site, it only ever reaches its one keyspace. A resolvable site key is always a valid one ([a-z0-9._-], per is_valid_site_key). The reserved namespace leads with the gossip unit-separator \x1f, which is outside that charset, so it can never equal any preview site key ⇒ no tenant is ever scoped to it. switchboard holds kv_secret and can AUTH as any site string, so it alone addresses the namespace.

Cross-node dedup unaffected: all nodes' daemons AUTH as the same reserved site over the same gossip-replicated store, and the per-commit key (analyze:verdict:<repo>:<pr>:<sha>:<cfg_hash>) keeps distinct previews from colliding within it.

New tests: verdict_namespace_is_unreachable_by_any_tenant (reserved site is not a valid site key; disjoint from real site keys) and all_previews_share_one_verdict_namespace (fleet-wide dedup preserved). VerdictCache dropped its preview-site parameter. 306 tests green; fmt + clippy -D warnings clean. Still do not merge / do not deploy.

@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:12 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:12 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:12 Inactive
…ment

The blocked-preview comment interpolated f.file and f.rule_id RAW inside
backtick code spans, and sanitize_cell (applied only to f.message) never
neutralized backticks. f.file is fully attacker-controlled — it is a filename
inside the PR and ephpm's SARIF artifactLocation.uri is emitted without
percent-encoding, so a newline/pipe/backtick in the name flows through
verbatim. A PR shipping a file named e.g. `x<newline>## Approved
[merge](...)<newline>.php` with malicious PHP is correctly BLOCKED, but
rendering the comment injected attacker markdown (heading/link/table breakout)
into switchboard's trusted-identity sticky comment — a reviewer-phishing
spoof.

Fix: sanitize_cell now also replaces the backtick (-> apostrophe); a new
sanitize_code helper is applied to the code-span fields file and rule_id.
Message keeps sanitize_cell. Cap unchanged.

Regression test blocked_comment_neutralizes_a_malicious_filename builds a
Finding whose file carries a newline, pipe, backtick and markdown, renders the
block comment, and asserts: one table row per finding, balanced backticks (no
span left open), and no active injected markup.
@luthermonson

Copy link
Copy Markdown
Contributor Author

🔒 Comment-injection fix (commit dc29511) — addresses the "comment rendering injection-safe" review finding.

Bug: format_block_comment interpolated f.file and f.rule_id raw inside backtick code spans, and sanitize_cell (only ever applied to f.message) didn't neutralize backticks. f.file is fully attacker-controlled (a PR filename; ePHPm's SARIF artifactLocation.uri is unencoded, so newline/pipe/backtick pass through). A PR shipping a file like x<newline>## Approved [merge](http://evil)<newline>.php with malicious PHP is correctly blocked, but rendering the comment injected the attacker's markdown into switchboard's trusted-identity sticky comment (heading/link/table-breakout spoof — reviewer phishing).

Fix (no redesign):

  • sanitize_cell now also replaces `' (on top of \n/\r → space and |\|).
  • New sanitize_code helper for the two code-span fields; applied to f.file and f.rule_id. f.message keeps sanitize_cell. The :line is our own u64, untouched. Cap (MAX_COMMENT_FINDINGS = 10) unchanged.

Regression test blocked_comment_neutralizes_a_malicious_filename: builds a Finding whose file carries a real newline, a |, a backtick, and markdown ([merge](...), ## heading), renders the block comment, and asserts (a) exactly one table row per finding (newline didn't break the row), (b) backticks are balanced (no code span left hanging open), (c) no active injected markup (\n## Approved absent; the rule_id backtick can't close its span).

307 tests green; fmt + clippy -D warnings clean. Verdict/reason strings (switchboard-controlled) left as-is per the review scope. Still do not merge — over to you to verify the rendering and merge.

@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:23 Inactive
@ephpm
ephpm Bot temporarily deployed to preview-pr-34 September 13, 2026 21:23 Inactive
@luthermonson
luthermonson merged commit f21f658 into main Sep 13, 2026
4 checks passed
@luthermonson
luthermonson deleted the feat/analyze-gate branch September 13, 2026 21:30

This branch was previously deployed

1 inactive deployment
preview-pr-34 dc295112 Deployed Sep 13, 2026 by ephpm[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant