feat(providers): add copilot_cli semantic-scan provider

[Yoseph-Zuskin]

Add copilot_cli semantic-scan provider

Problem

SkillSpector ships CLI providers for Claude, Codex, Gemini, and OpenCode,
but none for GitHub Copilot, so Copilot users get static-only scans
(llm_available stays false and the semantic analyzers are skipped) unless
they provide an OpenAI or Anthropic API key.

Fixes: #8

Approach

• New copilot_cli provider mirroring the merged opencode_cli shape:
  providers/copilot_cli/{provider.py,__init__.py}, registry entry,
  SKILLSPECTOR_PROVIDER=copilot_cli selection, provider_name() label,
  CLI help text.
• Transport is flags + piped stdin: copilot -s --no-ask-user with no -p
  flag — the prompt is piped to stdin by run_agent_cli (verified by nonce
  round-trip; -p "" is rejected, so stdin is the cleaner path).
  Untrusted content never reaches argv; list form throughout, shell never
  invoked. Windows hostile-prompt roundtrips covered by test.
• Least privilege from day one (not a caveat): --available-tools
  names a fixed implausible tool so the model is offered nothing usable
  (verified live: file-creation refused, no side effect), plus
  --deny-tool shell,write belt-and-braces in the documented
  Kind(argument) form (deny wins over allow). Never --allow-all*.
• --model <label> validated and forwarded only when SKILLSPECTOR_MODEL
  is set; max_output_tokens accepted for CliSpec uniformity and ignored.
• Auth probe is copilot --version with a 15s timeout, scrubbed env,
  fail-closed: non-zero exit, unparseable output, or anything but the
  pinned 1.0.86 fails. No status subcommand exists; authentication works
  via login session or one of COPILOT_GITHUB_TOKEN/GH_TOKEN/
  GITHUB_TOKEN (preserved deliberately by _prepare_copilot_env, which
  drops every other COPILOT_* and forces COPILOT_AUTO_UPDATE=false).
• No per-call preflight (no per-invocation policy to verify, unlike
  opencode's config layers) and no model registry (CLI fallback, per the
  registry-split lesson).
• Deliberate exception worth flagging: _prepare_copilot_env preserves
  COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN through the scrub
  (everything else COPILOT_* is dropped). Rationale: these are the CLI's
  documented headless auth path — without them, token-only CI setups (where
  GITHUB_TOKEN is often the only credential) cannot use the provider at
  all, and the issue being resolved is precisely keyless operation. The
  model itself gets no tools, so it cannot read the environment; the CLI
  redacts these variables from its own output by default. If reviewers
  prefer login-only, the fallback is a one-line change (extend the scrub,
  document fail-closed for token setups).
• Docs trio updated in the same PR (README provider table, .env.example,
  docs/DEVELOPMENT.md); provider tests live in tests/provider/
  per repo convention.

Verification

• tests/provider/test_copilot_cli.py: 41 passed, 2 POSIX-skipped
  (TDD: argv, auth, parser, wiring, registry label, adversarial).
• tests/unit/test_agent_cli.py registry loop extended; adversarial
  fake-host checks (deny posture + hostile env) verified via a Windows
  stand-in run: policy held, zero markers.
• Provider-adjacent suites (test_providers, test_new_providers,
  test_constants, test_llm_utils): 375 passed / 14 skipped total.
• ruff check + ruff format --check: clean on all touched files.
• Live single-skill probe (Copilot CLI 1.0.86, Copilot Free, CLI-default
  model): llm_available: true, 3/3 semantic calls succeeded,
  risk 0/LOW, 0 findings.

Sample

Probe scan of one small skill returned
risk: score 0 / LOW / SAFE, llm_calls_attempted: 3,
llm_calls_succeeded: 3, zero findings.

Risks

• Copilot Free budgets agent requests monthly; bulk scans will exhaust
  them — probes only, no bulk scanning on Free.
• --available-tools fixed-name posture depends on unknown names staying
  inert (verified on 1.0.86 behavior: silent acceptance); the version
  gate pins exactly 1.0.86 so any CLI behavior change fails closed first.
  The exact pin is deliberate (matches the merged opencode_cli policy):
  the tool-deny behavior was verified against this release, and a silent
  sandbox change must never pass unnoticed. Re-verification per Copilot
  release is the known maintenance cost.
• DCO: all commits carry Signed-off-by (maintainer: verify on push).

[rng1995]

[SkillSpector Review]

Reviewed exact head 6bb8dc5a6af1f8e59d4967f2e8133fd00d02bfa6.

The provider wiring, exact-version gate, stdin transport, output bounds, environment filtering, and zero-tool allowlist are well covered. One trust-boundary gap remains: the invocation leaves Copilot CLI's normal custom-instruction discovery enabled while deliberately preserving the user's home/login context. Ambient global or user instructions can therefore alter semantic-security judgments even though COPILOT_CUSTOM_INSTRUCTIONS_DIRS is stripped. Disable custom instructions explicitly for this provider and add an end-to-end argv/isolation regression before enabling it.

[yashrajp22]

Two issues remain in the Copilot safety boundary: inference bypasses the exact-version check, and ambient user/plugin lifecycle hooks remain enabled.

Validation: fresh base/head wheels with verified package identity; selected tests passed (324 base, 367 head; 10 skipped each); all 48 no-LLM corpus runs matched across source/wheel and base/head, with nine samples retaining partial reports. Synthetic unsupported-version checks reproduced prompt delivery through both public scans and direct completion. The real 1.0.86 binary reported missing authentication as expected, and the scan retained two PE3 findings while explicitly marking semantic analysis failed.

The hook finding is based on the pinned runtime source. The credential-free marker check was inconclusive; authenticated inference and native hook/tool/MCP behavior remain unverified. Corpus parity is not a global accuracy claim.

[yashrajp22]

Could we run the exact-version check as a completion preflight here? Normal scans and direct complete() calls bypass auth_check. With a synthetic binary reporting 9.9.99, both source and installed-wheel scans sent all three semantic prompts before calling --version; direct completion never called it. That lets an unsupported runtime receive scan content despite the stated 1.0.86 policy. Please reject it before passing stdin and cover the public scan path.

[Yoseph-Zuskin]

Done — _preflight_copilot_policy is now the copilot CliSpec preflight, so [binary, --version] runs under the isolated child env on every completion and anything but exactly 1.0.86 raises before stdin is written. This covers both paths you named, since public scans and direct complete() both funnel through run_agent_cli. Cost is one local no-inference subprocess per call. Tests: synthetic 9.9.99 is rejected with stdin-never-delivered asserted, plus nonzero/timeout/child-env cases.

[yashrajp22]

Could we isolate Copilot settings and disable user/plugin hooks here? In the pinned 1.0.86 source, session creation loads user and installed-plugin hooks through the retained HOME/COPILOT_HOME. The tool allowlist and deny rules govern model tools; they do not disable lifecycle command hooks, which can run with scanner privileges and receive prompt content. A temporary working directory and --no-custom-instructions do not close this path. Please preserve authentication separately and cover this with a real harmless-hook check.

[Yoseph-Zuskin]

Done, with one deviation from the suggested shape that I'd like to be transparent about. I first implemented exactly what you asked — HOME/USERPROFILE/COPILOT_HOME redirected to per-invocation temp dirs — then probed it live and had to revert: 1.0.86 silently refuses inference (exit 0, empty output, nothing created) under any redirected home, including a byte-identical copy and a short-path alias of the real dir, while the same calls succeed with homes untouched. I could not find a redirect the CLI accepts, so isolation would brick the provider rather than harden it.

What landed instead is enforcement by absence: the new _preflight_copilot_policy (already added for the version gate) now also audits the resolved copilot home and raises fail-closed when installed-plugins/ is present and non-empty. Plugins are the documented hook vector (copilot plugin --help: skills, agents, hooks, MCP, LSP), and I verified locally that 1.0.86 exposes no hook-disable flag — so with no hook material on disk, no hooks load. COPILOT_HOME passthrough is restored, keeping both login-session and token auth working; verified live on real 1.0.86 (audit passes on an empty plugin tree, inference succeeds) plus 4 new audit unit tests (tests/provider/test_copilot_cli.py::TestAuditCopilotHome: nonempty/empty/missing/default-path).

To keep this from being re-litigated per provider, I also added a "Provider CLI validation expectations" section to CONTRIBUTING.md (separate commit, revertable on its own) codifying the bar from this thread and #536: exact-version preflight before stdin on every completion path, no-hook-material enforcement (isolation where usable, presence-refusal otherwise), adversarial fake-host tests, synthetic-version gate tests, no silent fallbacks. If you know a hook path outside installed-plugins/, I'll extend the audit to cover it.

[rng1995]

[SkillSpector Review]

Re-reviewed exact head c9e001f29c7108fd3ecc23f7b6c4f15cac6fa53c, including the complete diff, provider/transport call path, tests, previous reviews and author replies, and CI.

Previous findings:

• Addressed: custom-instruction discovery is explicitly disabled with --no-custom-instructions; built-in MCPs and auto-update are also disabled, with argv coverage.
• Addressed in implementation: the exact-version check is now a registered per-completion preflight, invoked before Popen/stdin delivery, including direct complete() calls. The unsupported-version test currently calls the helper directly; please also exercise the public completion path so a future registry/wiring regression cannot bypass the gate unnoticed.
• Still open: user/plugin hook isolation is only partially addressed. Rejecting nonempty installed-plugins/ does not reject user hook files or inline settings hooks. The inline finding supplies the concrete non-plugin paths requested in the author reply.

I inspected (did not execute) package/app.js and the version metadata from the official Copilot CLI 1.0.86 release, asset github-copilot-1.0.86-darwin-arm64.tgz. createNativeHookSession loads user settings and passes both settingsJson and userHooksDir to hookSessionCreate before adding plugin hooks separately. This is pinned-runtime evidence, not an assumption from newer documentation.

All six reported CI checks pass. Contributor code/tests and authenticated Copilot inference were not executed during this review; the remaining finding is based on source inspection. Approval remains blocked on closing and testing the non-plugin hook paths.

[rng1995]

[SkillSpector Review]

Re-reviewed current head 05f3e3236ea5eae91ffa611faf1df4b5ba239a52, both follow-up commits, the current provider/transport code and regression tests, prior discussions, and CI.

The previous direct-home cases are now addressed: the audit rejects user hooks/*.json, inline settings.json hooks, unreadable settings, and missing explicit homes; the temp-directory tripwire and transport-level rejection test are also present. The custom-instruction/MCP restrictions and per-completion exact-version preflight remain wired in. I also independently checked the official Copilot 1.0.88 launcher: I am not requesting changes solely because auto-update was re-enabled.

One remaining user-hook path prevents approval: the new audit observes the home before Copilot's inference startup migrates legacy XDG hook material into it. The --version subprocess exits before those migrations, so it cannot make the subsequent home audit sufficient. This is a deterministic startup-order issue, not a hypothetical concurrent modification. The inline comment gives the affected configuration, exact pinned-runtime source path, and required correction.

Evidence: read-only inspection of package/app.js from the official Copilot CLI 1.0.88 release, asset github-copilot-1.0.88-darwin-arm64.tgz. The version branch returns before tOn(); tOn() calls X0n for XDG config/state migrations; normal session creation subsequently loads the destination home's hooks.

All six reported CI checks pass. Contributor code, tests, and downloaded Copilot code were not executed locally; this finding is source-traced. No merges or thread-resolution changes were made.

[rng1995]

[P1] Prevent startup migration from introducing hooks after the audit

With COPILOT_HOME unset, XDG_CONFIG_HOME set, no ~/.copilot/hooks directory, and a user hook at $XDG_CONFIG_HOME/.copilot/hooks/example.json, this audit passes: _scrub_env / _prepare_copilot_env retain XDG variables, but only the current ~/.copilot tree is inspected. In the pinned 1.0.88 package/app.js, the --version branch exits before tOn(). The subsequent inference startup calls tOn() / X0n() to move the XDG hooks directory into ~/.copilot, then createNativeHookSession loads that directory. Thus user hooks become loadable after this check and outside the tool allowlist, even without a race or administrator-owned policy. Please neutralize these migration inputs in the child environment or verify all migration sources before starting inference, and add a transport-level regression with a hook-free destination home plus hook material in the inherited XDG source. The existing tests place hooks only in the already-resolved home, so they do not exercise this path.

[Yoseph-Zuskin]

Done. The startup migration path is audited, not just the home: _audit_copilot_home now inspects both the resolved home and the XDG migration source ($XDG_CONFIG_HOME/.copilot, defaulting to ~/.config/.copilot) through one shared _audit_hook_tree helper covering plugins, hook files, and inline settings in each tree. Explicitly-set-but-missing sources fail closed either way; same-tree resolution dedups to a single audit. No environment neutralization, so CLI behavior is unchanged for clean trees. Ten new tests (explicit/default/missing/clean/falsy-hooks/empty-string/same-tree plus a transport-level end to end with clean COPILOT_HOME and dirty XDG tree asserting rejection before prompt delivery, no invocation marker). Live validation on a real-home copy: rejected with the migration-source message, marker absent. Suite 73 passed / 4 skipped, ruff clean.