Skip to content

Add a CI workflow that runs the skill-harness suite — the guards from #28 currently only run locally #33

Description

@oschaaf

Raised in review of #28: after that PR, pnpm test runs the schema-digest drift check, the coverage assertions, and the vendored-artifact pins — but nothing automated runs pnpm test. The repo has no .github/workflows/ (the only checks on PRs are org-level CodeQL default setup), so every guard added in #28 holds only when a human runs the suite locally.

Suggested work

A small workflow (PR + push to main) that runs, from skill-harness/:

  1. pnpm install --frozen-lockfile
  2. the unit suite (node --import tsx --test 'src/__tests__/**/*.test.ts' until pnpm test works again)
  3. biome check --error-on-warnings and tsc --noEmit
  4. node scripts/generate-schema-digest.mjs --check from the repo root (also covered by the suite, but a named check line is worth having)

Prerequisite: #30. pnpm install --frozen-lockfile currently fails on pnpm 11 (overrides placement), and a CI that installs unfrozen defeats part of the point. Fix #30 first or in the same PR, then wire the workflow.

Explicitly out of scope: the Tier-2 bench (pnpm bench) — it calls a model API and needs a key + cost decision; the workflow should run the offline suite only.

Once green, update README.md / CLAUDE.md wording from "pnpm test is what enforces it" to point at the workflow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions