Conversation
`--tests` appears exactly ONCE in every other workflow: ci.yml's BSE-17 quick
tier, scoped to the crates a PR touched. `make coverage` — the only other
whole-workspace run — is `--lib`. nightly.yml runs no tests at all. So 654 test
binaries have never been LINKED in CI, let alone run, and a binary that never
links cannot fail, flake, or appear in a report.
merge-queue full tier 82,217 tests / 73 binaries job 103868350604
this job 98,621 tests / 727 binaries 1141 s run, measured
NO BASELINE, HARD FAIL ON DAY ONE. The first draft of APR-DARK-TARGETS-001
proposed a shrink-only ratchet. A 4-lane agy quorum refused it and the objection
is mechanical: `--update` writes a file and needs a COMMIT, and a scheduled
workflow has no PR and no author. Every other ratchet here is maintained by a
human inside a PR BECAUSE it runs in one. A cron's baseline would only ever be
edited by someone who already noticed the job was red — the exact population a
ratchet exists to stop relying on. So the 37 failures are FIXED first, in five
PRs (#3250 34 rows, #3248, #3238, #3253), and this ships with nothing to
maintain.
NOT ON gx10, and the mechanism is measured rather than declared: the build
produces a 39 GB target dir (808 binaries), gx10 is the smallest of the three
Linux hosts at 916 GB against intel 3.6 TB and yoga 935 GB, and it hit ZERO BYTES
FREE on 2026-09-14 — dequeuing a PR from merge-queue position 1 (#3247). `X64` is
the exclusion: over 25 runs, no job requesting X64 has ever landed on gx10. No
new label.
Four things this job refuses to do quietly, each earned by a defect already paid
for here:
cargo_step.sh --self-test FIRST a sweep of 727 binaries is exactly where an
unclassified cargo death reads as 650 defects
preflight >= 120 GB free fail ENV: BEFORE the build, not half-way
through it — that is how #3247 presented
--no-fail-fast plain cargo test stops at the first failing
BINARY; seven rounds went into four tests
vacuity floor, 600 binaries a selection that built nothing reports zero
failures, which reads exactly like a pass
and it reclaims the 39 GB in-job under `always()`, because the host reaper's
critical mode already cannot reclaim under keep=3 (#3247) — leaking there
compounds a known-unfixed defect.
`timeout-minutes: 90` is deliberately generous and is NOT derived. BSE-05 wants
T = max(15, ceil(1.5*p99), p99+20) over the job's own history and this job has
none; the only reading in hand is 21 min on a 48-core box with a warm sccache,
and a clean-room runner measured 2.34x slower per test on the same workload. That
extrapolates to ~51 min — an extrapolation, not a measurement. Re-derive from
three real runs.
Also carries the selector-touch PR trigger from the spec: ci_test_tier.sh,
gate_touched_crates.sh, tree_reader_tests.txt and ci.yml. A change to how the
tier chooses what to run is exactly when the dark set moves.
Verified: yaml parses; check_no_hosted_runners, check_runner_labels,
check_perf_concurrency_groups, check_ci_reusable_workflow_pinned,
check_no_timing_in_required, check_cargo_install_private_root,
check_hardcoded_paths, check_guards_are_wired and
check_tool_versions --audit-workflow all pass on the new file.
Refs #3239
Pmat-Ticket: PMAT-1098
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
§13.11 rung 1 — quorum shadow verdict Shadow mode: this records a verdict and merges nothing. A refusal |
…ack out
The operator's challenge was right and I had argued from assumption: slowness is
what a nightly is FOR, and `.config/nextest.toml` already answers HANGING with
`slow-timeout = { period = "60s", terminate-after = 20 }` so "a genuine hang dies
at 20 minutes WITH A NAME". `check_examples_run` in aprender-orchestrate even
sidesteps stdin the obvious way: `cargo run --example X -- --help`.
So I wrote the clause. Then ran it on a random sample of five:
HANG aprender-core::grid_search_tuning
HANG aprender-serve::imp_700_realworld_verification
exit101 aprender-serve::debug_q4k_gemv
ok aprender-simulate::reproducibility
HANG aprender-serve::bench_barrier_overhead
THREE OF FIVE ran past 60 s on `--help`. These are not CLIs — they are compute
demos that ignore argv and do the work. `--help` does not make them exit, and
"hang" is the wrong word: they are RUNNING. A per-example timeout would report a
working example as a FAILURE, at 980 rows.
Five is a small sample and the ratio is not the claim; the KIND is, and the three
names are self-evidently long-running demos.
So "every `cargo run --example`" cannot be implemented as the spec words it
against this workspace. Two shapes that could work are written into the header
for whoever decides: redefine as "starts and does not crash within N seconds"
with timeout treated as PASS (~2 h at N=10, so it belongs in #3255's
full-surface lane, not this 5-minute one), or have examples DECLARE runnability.
Clause 3 — every example BUILDS, 980 targets, 83 s — already catches the common
rot, which is compile breakage. That is the part that is cheap and unambiguous,
and it stays.
Also recorded: `check_examples_run` is worth a ticket of its own. Its only
callers are its own tests, and it fails OPEN — a non-zero runtime exit counts as
a pass because "--help exiting non-zero is OK", so a crash passes.
The feature matrix stays out and that one is genuinely undefined: cargo-hack is
not installed and nothing here references --each-feature, --feature-powerset or
any matrix tooling. No definition and no tool.
Pmat-Ticket: PMAT-1098
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
No activity since 2026-09-17 13:47Z; 2 release trains have been cut since (v0.68.2, v0.68.1). Age is the only input to this sweep — it is not a judgement on the work, and a train label such as What happens next: if one more train passes while this is still labelled To clear it: push, rebase, or say on the thread what it is waiting for. Any of the three removes the label at the next sweep. If it is blocked on something external, name that here — a blocker with an owner is not sprawl, and it stops the clock. |
|
closed by 0.70 batch triage: no quorum receipt and no live owner; branch kept, reopen with an owner Authority: per operator order (quoted) "yes, clearn up. no sovereign repo can have triage, ticket, branche issues and arbiter shoudl be doing this! tell that agent to get to work" + cop (aprender-cf) ruling on the close criteria (a draft, or >7d old with no live owner, and no quorum receipt). The operator did not select this PR individually. |
Implements the cadence in #3249.
What it runs, and why nothing else does
--testsappears exactly once in every other workflow —ci.yml's BSE-17 quick tier, scopedto the crates a PR touched.
make coverage, the only other whole-workspace run, is--lib.nightly.ymlruns no tests at all.654 test binaries have never been linked in CI, let alone run. A binary that never links
cannot fail, cannot flake, and cannot appear in a report.
No baseline. Hard fail on day one.
The first draft proposed a shrink-only ratchet. A 4-lane agy quorum refused it, and the objection
is mechanical rather than stylistic:
--updatewrites a file and needs a commit, and a scheduledworkflow has no PR and no author. Every other ratchet here is maintained by a human inside a PR
because it runs in one. A cron's baseline would only ever be edited by someone who already
noticed the job was red — the exact population a ratchet exists to stop relying on.
So the 37 failures are fixed first and this ships with nothing to maintain.
Not on gx10 — and the mechanism is measured
The build produces a 39 GB target directory (808 binaries). gx10 is the smallest of the three
Linux hosts — 916 GB against intel's 3.6 TB and yoga's 935 GB — and it hit zero bytes free on
2026-09-14, dequeuing a PR from merge-queue position 1 (#3247).
X64is the exclusion: over 25 runs, no job requesting X64 has ever landed on gx10 (it isARM64). No new label.
Four refusals, each earned by a defect already paid for here
cargo_step.sh --self-testfirstENV:before the build, not half-way through — that is how #3247 presented--no-fail-fastcargo teststops at the first failing binary; seven rounds went into four testsIt reclaims the 39 GB in-job under
always(), because the host reaper's critical mode alreadycannot reclaim under
keep=3(#3247) — leaking there compounds a known-unfixed defect.The timeout is honest about being a guess
timeout-minutes: 90, deliberately generous and not derived. BSE-05 wantsT = max(15, ceil(1.5·p99), p99+20)over the job's own history and this job has none. The onlyreading in hand is 21 min on a 48-core box with a warm sccache; a clean-room runner measured
2.34× slower per test on the same workload, extrapolating to ~51 min — an extrapolation, not a
measurement. Re-derive from three real runs and tighten.
Also carries the selector-touch trigger
ci_test_tier.sh,gate_touched_crates.sh,tree_reader_tests.txt,ci.yml. A change to how thetier chooses what to run is exactly when the dark set moves.
Verified: yaml parses;
check_no_hosted_runners,check_runner_labels,check_perf_concurrency_groups,check_ci_reusable_workflow_pinned,check_no_timing_in_required,check_cargo_install_private_root,check_hardcoded_paths,check_guards_are_wiredandcheck_tool_versions --audit-workflowall pass on the new file.Refs #3239
no-close: #3239 stays OPEN until this is undrafted and has run green three times — a workflow that
has never fired is a claim, not a gate.
🤖 Generated with Claude Code