Skip to content

ci(diet): coverage on tags only, perf-gate table path-scoped, model suites nightly, mutants only on .rs, gate's skipped rule tightened (PMAT-3676) - #3688

Open
noahgift wants to merge 5 commits into
mainfrom
PMAT-3676-ci-diet
Open

noahgift wants to merge 5 commits into
mainfrom
PMAT-3676-ci-diet

Conversation

@noahgift

@noahgift noahgift commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Refs #3676 #3668 #3658. Not armed. Per the batching process it rides the next batch. It edits .github/workflows/ (ci.yml, coverage-nightly.yml, guards-nightly.yml). Noah approved all five items directly; the operator's words are quoted on #3676.

keep-open: #3676 stays open until done_when item 1 (≥40% fewer intel runner-minutes per 24 h) is measured after merge.

# Item Change Evidence
1 coverage on tags only skip_coverage: true on the sovereign-ci call. coverage-nightly.yml also runs on push: tags: ['v*'] (COV_FLOOR), with per-ref concurrency. The job gated nothing (no coverage_min) and measured 0 tests (--lib on the facade root, per the existing NOTE). It cost 237 s per run on main 35563537942.
1′ the rmedia trap Every consumer enumerated. The dogfood runs make coverage-check and produces its own number; the steward reacts only to failure; the tests read an embedded string or the Makefile; prepare-release.sh runs llvm-cov locally. None reads CI's number. The one output lost is the advisory Codecov upload. NEW check_coverage_has_producers.sh covers R1–R4 in 10 rows, and each broken link turns RED.
2 book No change. Already path-filtered. book-contracts watches crates/aprender-core/src/** because the chapters run aprender-core code, and check_workflow_path_filters.sh RULE 2 requires that.
3a perf-gate table The explicit guard-tree step (405 s) is replaced by NEW check_perf_gate_selftest_scoped.sh. It runs only when the diff vs origin/main touches the derived scope: every $ROOT/ path perf_gate.sh reads, followed transitively (8 paths). Otherwise it prints a SUMMARY skip line. An unknown comparand means RUN. It runs whole every night. 11 rows. Mutants that each go RED: not transitive, renames hiding the old side, unknown → skip, prefix-string match.
3b model suites The PR step keeps only falsification_spec_v10_tests, the SATD ratchet vs origin/main and the one property a diff can change. The stress and gpu_state suites run nightly, as they already did. check_model_tests_wired: all 3 still wired.
4 mutants The diff step emits has_rs (--no-renames). Image pull, cargo-mutants install, the run and the upload are gated on it.
5 gate On a pull_request, mutants must succeed. skipped passes only on other events, where its own if: skips it. The old rule passed a PR whose mutants never ran. NEW check_ci_gate_mutants_rule.sh executes the gate's own marked block for 10 (event, result) pairs. The old rule, planted, goes RED.

Not done, stated: T-4 waiting on the tag's coverage run. That is an autopilot change, deliberately not made on the 0.69 cut day. Today a floor breach reds the tag's coverage run but does not stop the cascade.

Gates:

Update 2026-09-21 ~15:45Z (aprender-62, cop ruling): the ci / gate fix, coverage_on: tag on the merged paiml/.github#77 pin

This PR's first spelling, skip_coverage: true on sovereign-ci@70e51ec, turned ci / gate RED on every PR: that reusable's gate counts a skipped coverage job as a mandatory failure ("coverage did not succeed (result: skipped)"). This PR's own run 35585906113 showed it (it was cancelled unread), and batch-1 #3707 hit it and left this row out. Head a738b926c, a fast-forward from e2d12bdd1 that carries the seated f7b25a6c6 plus:

Proof required before re-folding into batch-2: this PR's own CI green, with the gate logging coverage: NOT MEASURED and roadmap-fragment-parity: NOT-ACTIVE.

keep-open: #3676 stays open until the CI diet is measured on main after this row lands (batch-2); #3668 and #3658 are cited for context and close on their own rows; #3709 is the follow-up migration (stop tracking roadmap.yaml) that arms #73's check.

🤖 Generated with Claude Code

…uites nightly, mutants only on .rs, gate's skipped rule tightened (#3676)

Refs #3676 #3668 #3658

Operator 2026-09-21: aprender releases as fast as possible, intel idle;
"YES, coverage on tags release only." All five workflow items approved by
Noah directly (CLAUDE.md check-in for .github/workflows edits).

1. COVERAGE -- ci.yml `skip_coverage: true` on the sovereign-ci call. That job
   gated nothing (no coverage_min) and measured nothing (`--lib` on the facade
   root: 0 tests, per the existing NOTE), at 237 s per run (main 35563537942).
   coverage-nightly.yml now also triggers on `push: tags: ['v*']` (COV_FLOOR
   via `make coverage`), concurrency per ref so a tag run and the nightly do
   not cancel each other. Every consumer enumerated first (the rmedia trap
   aprender-04 relayed): the pre-publish dogfood's `make coverage-check`
   (produces its own llvm-cov), the queue steward (reacts only to `failure`),
   two tests reading an embedded string / the Makefile, a legacy
   prepare-release.sh (local llvm-cov). None reads CI's number. The one output
   lost: that job's advisory Codecov upload.
   NEW scripts/check_coverage_has_producers.sh asserts the chain: R1 `make -n
   coverage-check` reaches llvm-cov; R2 dogfood runs it (non-comment); R3
   coverage-nightly's `on:` block has schedule AND tags [v*] (comments and
   other top-level keys don't count); R4 numeric COV_FLOOR. 10 rows, each
   link broken -> RED; missing file -> ENV 2.
   NOT done here, stated: making T-4 WAIT on the tag's coverage run is an
   autopilot change, deliberately not made on the 0.69 cut day.

2. BOOK -- no change. book.yml and book-contracts.yml are already path-filtered;
   book-contracts watches crates/aprender-core/src/** because chapter examples
   run aprender-core code, which check_workflow_path_filters.sh RULE 2 requires
   ("a gate that runs code must watch the code it runs"). Narrowing it would
   break that rule.

3a. PERF GATE TABLE -- the explicit guard-tree step (405 s on main 35563537942)
   is replaced by NEW scripts/check_perf_gate_selftest_scoped.sh, dispatched by
   guard_tree.sh --no-cargo: it runs `perf_gate.sh --selftest` only when the
   tree diff vs origin/main touches its DERIVED scope -- every $ROOT/ path
   perf_gate.sh reads, followed transitively through the scripts it calls (8
   paths today) -- and otherwise prints a SUMMARY skip line. Unknown comparand
   or a failed diff -> RUN, never skip. The table runs whole nightly
   (guards-nightly.yml + manifest, 405 s). 11-row case table; mutants RED:
   scope not transitive, renames hiding the old side, unknown -> skip,
   prefix-string matching. Registered in check_no_timing_in_required.sh's
   META_GUARDS (reads no clock).
3b. MODEL SUITES -- the PR step keeps ONLY falsification_spec_v10_tests: it holds
   the SATD ratchet measured against origin/main, the one property here a diff
   can change (and vacuous on the nightly, where HEAD == origin/main). The
   stress and gpu_state suites judge the whole tree; guards-nightly.yml
   already runs all three. check_model_tests_wired: all 3 still wired.

4. MUTANTS -- the diff step emits has_rs (`git diff --no-renames --name-only`);
   image pull, `cargo install cargo-mutants`, the run and the upload are gated
   on it. A diff with no .rs passes quickly instead of installing the tool.

5. GATE -- the mutants rule, between GATE-MUTANTS-RULE markers: on a
   pull_request mutants must SUCCEED (a no-.rs diff succeeds); `skipped` passes
   only on other events, where the job's own `if:` skips it. The old rule
   ("success/skipped both pass") passed a PR whose mutants never ran.
   NEW scripts/check_ci_gate_mutants_rule.sh EXTRACTS that block from ci.yml
   and executes it for 10 (event, result) pairs; its --self-test plants the old
   rule and it goes RED (3 rows); no block -> ENV 2.

Gates: guard_tree --no-cargo: all PASS except check_fleet_pv_shapes_gate.sh,
the known lambda-box fleet-pv condition (identical on main; fixed in #3669);
guard_tree_test 23/0; guards-are-wired, guards-nightly-manifest (8 entries),
no-timing-in-required, model-tests-wired PASS; bashrs 0 errors on the new
files, bashrs-gate PASS; actionlint: the same 7 shellcheck infos as main,
nothing new; roadmap aggregate idempotent; fmt, deny, contracts 1666.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3676",
 "head": "e2d12bdd1203666f133be3f766d908950e7f4ee0",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 3
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 3
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 6
  }
 ]
}

…, perf-gate smoke row, gate-rule self-test independent of ci.yml (#3676)

Refs #3676

Round 1 (e2d12bd) was FAIL/FAIL/PASS. The two FAIL lanes held the diff to
the ticket's text and were right on three counts:

1. Item 2 says the book jobs are path-filtered "and nightly always". They were
   filtered, but NO path-filtered workflow had a schedule. book.yml,
   book-contracts.yml and install-script.yml now carry a nightly cron
   (22:30 / 22:45 / 23:00 UTC, ~5h early for the #3292 dispatch delay; book.yml's
   pages deploy stays push-only by its own `if:`).
2. Item 5 says check_workflow_path_filters.sh is updated. It gains RULE 3: a
   path-FILTERED workflow must also trigger on `schedule:` with a cron -- the
   filter is a claim that nothing outside it can break the gate, and the
   nightly is what re-checks it (book.yml sat dark three months, per the
   guard's own header). scripts/lib/workflow_path_filters.py emits a SCHEDULE
   row. Self-test 4 -> 6: no schedule -> RED; `schedule: []` -> RED. On the
   tree before the crons: 3 workflows RED; after: PASS.
3. Item 3 says "The PR path keeps a smoke row" for the perf table. On a skip,
   check_perf_gate_selftest_scoped.sh now requires `perf_gate.sh
   --list-selftests` to exit 0 and enumerate >= 1 case (109 today, ~60 ms), and
   says so on the SUMMARY line. Three end-to-end rows (skip + smoke; in scope
   -> full table; a table that cannot enumerate -> RED). 14 rows; five
   mutants RED (not transitive, renames, unknown -> skip, prefix match, smoke
   removed).
Lane 1's other finding also held: check_ci_gate_mutants_rule.sh checked for
ci.yml before --self-test, so its self-test could not run without the
workflow. The check now sits on the bare-run path only.

Gates: guard_tree --no-cargo all PASS except check_fleet_pv_shapes_gate.sh
(the lambda-box fleet-pv condition, fixed in #3669); guard_tree_test 23/0;
bashrs 0 errors on every changed script, bashrs-gate PASS; actionlint: no
finding beyond main's shellcheck infos.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

noahgift commented Sep 21, 2026

Copy link
Copy Markdown
Contributor Author

Non-author measured seat: PASS (aprender-7a, 2026-09-21 11:20Z). I did not author PMAT-3676. Everything ran in /mnt/nvme-raid0/agent-wt/seat-3676 (detached, f7b25a6c6), after git fetch origin main.

# command claim measured
1 check_coverage_has_producers.sh / --self-test R1–R4 ok, PASS / 10 rows rc 0: ok R1 (make -n coverage-check reaches llvm-cov), ok R2 (dogfood runs coverage-check), ok R3 (coverage-nightly on schedule AND v* tags), ok R4 (numeric COV_FLOOR), PASS / rc 0 SELF-TEST PASSED: 10 rows
2 check_ci_gate_mutants_rule.sh / --self-test / GATE_RULE_WORKFLOW=/nonexistent … --self-test 10 rows; PR only success passes; off-PR skipped/success pass, failure/cancelled FAIL / old rule RED 3 rows + no-block ENV rc 2 / independent of ci.yml rc 0. Rows: 1 PR success→pass, 2 PR skipped→FAIL, 3 PR cancelled→FAIL, 4 PR failure→FAIL, 5 mg skipped→pass, 6 push skipped→pass, 7 dispatch skipped→pass, 8 mg success→pass, 9 mg failure→FAIL, 10 mg cancelled→FAIL / ok the OLD rule is RED: 3 row(s) + ok a workflow with no rule block is ENV rc=2 / rc 0 SELF-TEST PASSED
3 check_perf_gate_selftest_scoped.sh --self-test / bare 14 rows / SUMMARY skipped + smoke 109, PASS rc 0 SELF-TEST PASSED: 14 rows / SUMMARY perf_gate --selftest skipped: the diff vs origin/main touches 264 path(s), none of the 8 the table reads; guards-nightly.yml runs it whole; smoke: 109 case(s) enumerated, then PASS ✅
4 check_workflow_path_filters.sh --self-test / bare / RULE 3 control (main's book.yml checked out) 6/6 / 3 path-filtered, PASS / FAIL book.yml rc 0 (6/6) / scanned 21 workflow(s), 3 of them path-filteredPASS … runs nightly / rc 1 FAIL book.yml: path-filtered, but no \schedule:` trigger.`, restored ✅
5 guard_tree.sh --no-cargo all PASS except check_fleet_pv_shapes_gate 79 checks, 1 failed: only check_fleet_pv_shapes_gate.sh [run]. The 3 new guards ran as both [self-test] and [run], all PASS ✅
6 check_no_timing_in_required.sh PASS rc 0 PASS no timing assertion can reach a required status check.
mutation in ci.yml's rule block, pull_request:skipped) ;; before the pull_request:*) arm, then check_ci_gate_mutants_rule.sh rc 1, row 2 FAIL rc 1: FAIL row 2 pull_request skipped wanted 1, got 0. Rows 3/4/9 are still correct. Restored, with 0 tracked changes ✅

noahgift and others added 3 commits September 21, 2026 17:31
… closes-issue --self-test (provisional pin 5997b41)
…-tag

# Conflicts:
#	docs/roadmaps/roadmap.yaml
…— `coverage_on: tag` on the merged #77 pin, not `skip_coverage: true` (PMAT-3676)

`skip_coverage: true` on sovereign-ci@70e51ec made `ci / gate` RED on every
PR: that reusable's gate counts a skipped coverage job as a mandatory failure
("coverage did not succeed (result: skipped)"). #3688's own CI (run
35585906113) and batch-1 #3707 both hit it, and #3707 left this row out.

- sovereign-ci is pinned to paiml/.github f713290c86fcd70d6f26a0faab14e67e6713586f:
  #74's `coverage_on` input, plus #77, which keeps #73's
  roadmap-fragment-parity NOT-ACTIVE while roadmap.yaml is tracked at the
  base (aprender still commits it; #3709 migrates it).
- `coverage_on: tag` replaces `skip_coverage: true`, and ci.yml gets
  `push: tags: ['v*']`, the reusable's second required edit.
- coverage-nightly's v* tag trigger is removed, so ci.yml measures each tag
  once.
- check_coverage_has_producers.sh R3/R5 assert all of this. Self-test 16/16,
  including "skip_coverage: true instead of coverage_on -> RED". On f7b25a6
  it reports R3+R5 RED; after the change it PASSES.
- ci.yml now runs `check_pr_closes_issue.sh --self-test` beside `--body`
  (guard_tree skips it as wired-with-args, so its case table ran in no job;
  found by aprender-f0), and the comment claiming otherwise is corrected.

Local: guard_tree --no-cargo 82/0 (after merging main 225b2a9). The proof
before re-folding into batch-2 is this PR's own CI: green, with the gate
logging `coverage: NOT MEASURED` and `roadmap-fragment-parity: NOT-ACTIVE`.

Refs #3676 #3707 #3709 paiml/.github#77

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3688 head=a738b926c889c40c27c8e85b81c7b3b0e257bcfe verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant