Skip to content

feat(ladder): model gate judges cells (verb × thinking × context rung) from the measured inventory (#3712 row B1) - #3738

Closed
noahgift wants to merge 4 commits into
PMAT-3712-ladder-universe-inventoryfrom
PMAT-3712-cells
Closed

noahgift wants to merge 4 commits into
PMAT-3712-ladder-universe-inventoryfrom
PMAT-3712-cells

Conversation

@noahgift

@noahgift noahgift commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

#3712 row B1: the model gate judges cells (verb × thinking × context rung), derived from each model's measured inventory

Stacked on #3721 (row A). Scope: done_when 5 on #3712 (issue author, comment 5764495191) adds the cells matrix; done_when 1 is amended in comment 5764277522 (the qwen3-8b flip rides the 0.69.1 fold). The fragment quotes both verbatim. The base is PMAT-3712-ladder-universe-inventory, so this diff is B1 only.

Operator 2026-09-21, verbatim: "lets ensure this release has MORE than enough tokens for our needs (overdo it), and our verbs are dogfooded, i.e chat with and without thinking, ditto run, ditto code, etc,". A cell is (model, host, verb, thinking, context rung). This PR is the judge side. The producer side (B2, measuring the cells) depends on apr work in flight and is not here.

What the judge now requires (scripts/lib/model_ladder_cells.py, called from check_model_ladder.sh)

rule source
owed set = short rungs up to each model's context_length, plus long rungs for qwen35 and ONE representative per other arch (named in the contract), × cells.verbs × thinking modes cop 2026-09-21
thinking modes come from the template evidence: enable_thinking → both; an always-open <think> → on; otherwise off. The producer's own claim can't shrink it #3723
fit := weights + kv_bytes_per_token × tokens + workspace ≤ the host's total GPU memory; an unmeasured term keeps the cell owed in full operator "a", #3710
a fitting cell must pass: cuda, no fallback, rc 0, prompt_tokens ≥ rung, answer_chars > 0, think_closed when on #3712
a non-fitting cell must be an honest refused with required_bytes > total. A co-tenant refusal, an OOM, or a pass beyond the declared arithmetic (the arithmetic decides what is owed, so it must be right) is RED operator "a", MCL-INV-008
every owed cell passes on ≥ 1 required host; every arch has a representative that some required host holds #3710, cop
rungs come from evidence/release/context-rungs.json (apr-release-context-rungs/v1, #3715), one declaration shared with pv. A missing file, or a consumer without a number, is a named FAIL #3715
anti-shrink floors vs origin/main: the cells block (block, verbs, long-rung families and representatives) and context-rungs.json (rung ids, consumers), since that file is where the rungs live row A pattern

Measured on this head

  • check_model_ladder.sh --self-test: 47 cases, 0 bad (21 new cells cases, each red for its own reason). All mutants are killed: 3 judge, 4 producer-lock, and 11 cells-module mutants (missing-cell, cotenant, passes-nowhere, think-closed, fell-back, prompt-short, modes-evidence, no-representative, pass-beyond-fit, rungs-floor, family-long), each by its named case.
  • Measured for the families rule: Qwen3.5 GGUFs report general.architecture = qwen35 (not qwen2), so families: [qwen35] matches by arch. The fixture's qwen35 model owes every long rung, and the family-long mutant proves the rule is tested.
  • Round 1 of the quorum (1 PASS / 2 FAIL) found two real defects, both fixed: a non-fitting cell that passed was accepted, and the rung-id anti-shrink was vacuous once the rungs moved to context-rungs.json. The third finding (required: false) is covered by the issue author's done_when 1 amendment on P0 release-process: the train can publish with "most models working" — the model gate is a hand-picked ladder with OPTIONAL rungs, a skip reads as pass, and the dogfood can be inherited. Every Q4_K model × {lambda, gx10} × CUDA must be green or NOTHING ships #3712, which the fragment quotes verbatim.
  • guard_tree --no-cargo 76 checks, 0 failed.
  • pv validate ok · pv lint contracts/ PASS · census and contracts.nt unchanged · bashrs 0 errors · shell-lint ratchet PASS.

Where the gate runs

check_model_ladder.sh is declared only in Cargo.toml [package.metadata.dogfood] gates (the T-2 pre-publish dogfood). It is also listed in scripts/unwired_guards_baseline.txt: no PR workflow runs it. Its real run is RED until the producer writes cells and #3715 lands the rung file. That is the operator's release bar, not a PR red.

Not here

Refs #3712
keep-open: row B1 of #3712 only. Row B2 (the producer's cell measurement) and done_when 3 and 4 remain; the rung file lands with the pv shape; the qwen35 declared rung waits on the inspect fix.

🤖 Generated with Claude Code

…rung), derived from the measured inventory

Operator 2026-09-21: "lets ensure this release has MORE than enough tokens for our
needs (overdo it), and our verbs are dogfooded, i.e chat with and without thinking,
ditto run, ditto code, etc,". #3712 row B1, the judge side; stacked on row A.

- scripts/lib/model_ladder_cells.py (called from check_model_ladder.sh): the owed set
  is short rungs up to each model's context_length plus long rungs for qwen35 and one
  representative per other arch, x cells.verbs x the thinking modes its TEMPLATE
  evidence derives (#3723). fit := weights + kv_bytes_per_token*tokens + workspace <=
  the host's TOTAL GPU memory (operator "a", #3710). A fitting cell must pass (cuda,
  no fallback, rc 0, prompt >= rung, an answer, thinking closed); a non-fitting one
  must be an honest refusal; every owed cell passes on >= 1 required host; every arch
  has a held representative. Rungs come from evidence/release/context-rungs.json
  (#3715), one declaration shared with pv; missing or unsized is a named FAIL.
- contract: ladder.cells {verbs, long_rungs_for}; MCL-INV-008; FALSIFY-MCL-016. The
  cells block is an anti-shrink floor vs origin/main.
- case table 26 -> 45; 8 cells-module mutants, each killed by its case.

Refs #3712

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3712",
 "head": "7d7a338f301eb938632f567c3a94f5cbe9266f76",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 3
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 1
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 5
  }
 ]
}

…loor is the context-rungs file

Quorum round 1 on #3738 (1 PASS / 2 FAIL) found two real defects:
- a non-fitting cell that reported `pass` was accepted, contradicting MCL-INV-008
  (not fits => refused). The arithmetic decides which cells are owed, so a pass beyond
  it proves the arithmetic wrong: now a named FAIL.
- the rung-id anti-shrink read the cells block, which has no rungs since they moved to
  evidence/release/context-rungs.json: vacuous. The floor is now that file at
  origin/main (rung ids and consumers; absent at main = bootstrap).
The module also keeps only REQUIRED hosts' receipts itself instead of relying on the
caller. The third finding (qwen3-8b required: false) is answered by the issue author's
done_when 1 amendment on #3712, quoted verbatim in the fragment.

Case table 45 -> 47; cells-module mutants 8 -> 10, each killed by its case.

Refs #3712

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3712",
 "head": "bc37412ebda3b253e5dae7d95d2bd3fe55fd4231",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

…ent and done_when 5; title matches

Quorum round 2 on #3738 judged the cells matrix unrequested scope (the operator's bar
raise was a comment, not a done_when) and read the fragment title's "no Q4_K rung
optional" against the ruled required: false. The issue author amended #3712
(comments 5764277522 and 5764495191); the fragment quotes both verbatim, and the title
now states the amended criterion.

Refs #3712

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3712",
 "head": "234d7a96f55acc616413d61707fdffa1b52465df",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 1
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

…he arch it matches is measured

Quorum round 3 on #3738 (2 PASS / 1 FAIL): the FAIL claimed qwen35-family GGUFs report
general.architecture = qwen2, so families: [qwen35] never matches, and that the tests hold
no qwen35 model. Both are refuted by measurement: Qwen3.5-0.8B/4B report qwen35 (apr
inspect and a direct header read), and the fixture's q.gguf is arch qwen35 and owes every
long rung. The refutation is now executable: cells mutant family-long (the families check
deleted) is killed by red-cells-missing-cell. The fragment records the measurement.

Refs #3712

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3712",
 "head": "fd636b3d734cdc305c93a8aed1e773eb4c1bc5b9",
 "width": 3,
 "executor": "agy",
 "agreed": true,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 1
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 1
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

noahgift added a commit that referenced this pull request Sep 21, 2026
…— the vocabulary is S1's surface (#3745 S3)

Operator 2026-09-21: "the way to prevent testing leakage is tests are dervived from
surface and SHACL enforced...WE ARE NOT ENFORCING our interface AT ALL..it is hand coded
not derived". #3752 (#3745 part S3), stacked on S1 (#3749, surface::emit()).

crates/apr-cli/src/hand_list_guard.rs, a #[cfg(test)] module of apr-cli's lib:
- vocabulary = crate::surface::emit(): verbs are command path tokens and aliases; flags of
  a verb are its --long/-short/aliases plus global_args. Nothing here names a verb.
- an enumeration (S3.1 as amended): a literal sequence whose verbs, or flags of one verb,
  are >= 2 AND a strict majority of its items. Read from YAML, shell arrays (+= merged),
  for-in lists, verb case arms, comma/assigned strings, and bracketed string-literal
  lists in Python heredocs and Rust (merged across `+`). An argv, a script's own option
  parser and clap's help/version are not enumerations.
- scan set: the Cargo.toml dogfood gates, scripts/release/**, the dogfood scripts, the
  ladder and release-readiness contracts, the release-evidence extractor. No allow-list.
- runs in workspace-test (required) on every PR and in the pre-publish dogfood through
  `make coverage-check` (llvm-cov --workspace --lib; a failure is a NO-GO).

Measured: 4/4 green on main + S1; RED on #3738's ladder cells.verbs when dropped in.
S3.4 (the ladder/B1 migration to S2's derived cells) lands on S2.

Refs #3752 #3745

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

closed by triage (aprender-dd; cop aprender-cf approved): FOLDED, not abandoned. Per the release-process owner aprender-6c [8b6b78]'s ruling on #3712, this row was folded into the 0.69.1 squash batch #3772; its fold order lists it, and its cases are on main. The squash is why its head is not an ancestor of main. Branch kept.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant