Skip to content

feat(crux): AutoGluon becomes a CRUX competitor — category O, 24 contracts, 25 tickets, registry edit + mutation proof - #3395

Closed
noahgift wants to merge 8 commits into
mainfrom
crux/autogluon-category-O
Closed

noahgift wants to merge 8 commits into
mainfrom
crux/autogluon-category-O

Conversation

@noahgift

@noahgift noahgift commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

What

../autogluon (1.6.3) researched and admitted to CRUX as Category O — AutoML Parity, the way #3169 admitted linfa and burn as Category N.

Closes #3401 (the admission ticket). Refs #3370 (epic) and #3371#3394 (one issue per story).
keep-open: #3370 (epic) and #3371#3394 stay open — this PR files the tickets and their contracts; each issue closes when its story lands with a bound test. (This line was no-close: until the 2026-09-21 adoption; that marker is the #3400 class and check_pr_closes_issue.sh does not honour it.)
ont-delta: none — a competitor admission (new CRUX category, contracts, tickets, evidence); it moves no ont.* counter and resolves no known-red entry. README.md changes only through the generated CONTRACT_COUNT block and docs/specifications/ only in the CRUX subspec.

Diff inventory

  • contracts/crux-O-01..24-v1.yaml — 24 new contracts, registry: false, real equations, 51 LIVE-PENDING falsification gates, proof obligations, kani harness stubs, github_issue on each.
  • contracts/crux-competitive-research-ux-v1.yaml — 24 Category O story rows; coverage_intake 267 → 291 (partial 72 → 77, missing 152 → 171); evidence_sources.autogluon; autogluon reference.
  • crates/aprender-contracts/src/schema/validator.rsCRUX_COMPETITORS 14 → 15 (+ autogluon, with admission rationale).
  • crates/aprender-contracts/src/schema/crux_intake_tests.rsautogluon added to the corpus-vocabulary test and to the not-a-BEAT-pillar test.
  • scripts/crux_scaffold_contracts.pyCATEGORY_NAMES["O"].
  • docs/specifications/crux-competitive-research-ux-workflows.md — v2.2 → v2.3: §3 rows for Category N (never recorded there by feat(crux): linfa and burn become CRUX competitors — category N, 17 contracts, registry edit + mutation proof #3169) and Category O; §5 Category O table; §6 note that the YAML coverage_intake is the source of truth.
  • evidence/crux/autogluon/{README.md,api-surface.md,readme-verbs.txt,hello.sh} — the survey.
  • docs/roadmaps/entries/PMAT-3370.yaml, PMAT-3371..3394.yaml — 25 fragments; docs/roadmaps/roadmap.yaml regenerated by the aggregator.
  • README.md — CONTRACT_COUNT 1835 → 1866 (generated).

Absent on purpose: no library code, no CLI wiring, no test bodies. Every gate is LIVE-PENDING and says which test in which module promotes it.

The gap in one paragraph

AutoGluon's README is one code block: TabularPredictor(label="class").fit("train.csv", presets="best"); predictor.predict("test.csv"). aprender's automl/ is a hyperparameter tuner for one estimator the caller already chose, and time_series/ is one univariate ARIMA. The building blocks (encoders, calibration, SHAP/LIME/permutation importance, KFold/cross_validate, RF/GBM) exist; the predictor-level workflow (fit(label) → leaderboard → weighted ensemble → deploy; panel forecasting with quantiles and backtests) does not. 9 P0 / 9 P1 / 6 P2; MultiModalPredictor, autogluon.cloud, MLZero and Ray-parallel fits are CUT on the epic with reasons.

Verification

  • pv built from this tree validates 25/25 (24 new + master). The stale ~/.cargo/bin/pv rejects crux-O-01 with CRUX-002 — the delta proves the registry edit engaged.
  • Mutation-verified: deleting "autogluon" from CRUX_COMPETITORS turns competitor_registry_covers_the_corpus_vocabulary and the_real_crux_registry_rows_are_all_in_domain RED; restored, 20/20 green.
  • cargo test -p aprender-contracts --lib: 1526 passed, 0 failed.
  • roadmap_fragments.py aggregate --check: idempotent. Guards run locally with rc=0: roadmap fragment-required / ids-unique / sorted / diff-additive / completion-cited, contract test-binding and enforcement, shell-lint ratchet, hardcoded paths, readme claims, grep -q ratchet.

🤖 Generated with Claude Code

…racts, 25 tickets, registry edit + mutation proof

Research of ../autogluon (1.6.3 @ 77946149) as a competitive-research
source for aprender, landed the way category N landed linfa and burn
(#3169): the competitor is admitted to the CLOSED registry, every story
is a real contract with falsification gates, every story is a registry
row, and every missing/partial story has a GitHub issue and a roadmap
fragment.

What AutoGluon is, measured from the tree rather than recalled: three
predictors. TabularPredictor (65 public methods, 11 presets, 24 model
families of which 8 are tabular foundation models added in 1.4-1.6),
TimeSeriesPredictor (Chronos-2/Toto-2 pretrained, 30+ local/deep models,
16 metrics incl. WQL/MASE/RMSSE, auto backtesting since 1.5) and
MultiModalPredictor. Evidence under evidence/crux/autogluon/.

What aprender has, measured at eb262f8: automl/ is a single-estimator
hyperparameter tuner (TPE, grid, random, DE, TimeBudget, EarlyStopping);
time_series/ is one univariate ARIMA; encoders, calibration, SHAP/LIME/
permutation importance and KFold/cross_validate exist as building
blocks. No predictor-level fit(label), no leaderboard, no bagging,
stacking or greedy weighted-ensemble selection, no panel forecasting,
no quantile forecast metrics. The gap is the AutoML UX, not the
algorithms.

Category O — AutoML Parity — 24 stories: 9 P0 (the README hello-world:
fit(label), problem-type inference, presets, leaderboard, feature
pipeline, weighted ensemble, time budget, panel forecaster, quantile
metrics), 9 P1 (bagging, stacking, importance, threshold calibration,
deployment artifact, tabular foundation model, backtesting, local
baselines, pretrained forecaster), 6 P2 (refit_full, distill,
infer_limit, fit diagnostics, memory-aware fit, covariates).
MultiModalPredictor, autogluon.cloud, MLZero and Ray-parallel fits are
CUT on the epic with reasons.

  CRUX_COMPETITORS: [&str; 14] -> [&str; 15]   + autogluon

Not a BEAT pillar: aprender claims no pinned-benchmark win over
AutoGluon. Both registry tests that keep BEAT_INCUMBENTS and
CRUX_COMPETITORS apart are extended, not worked around.

Tickets: epic #3370, stories #3371-#3394, label pareto-autogluon.
Roadmap: 25 fragments under docs/roadmaps/entries/, roadmap.yaml
regenerated by the aggregator (idempotent check passes).

Spec: docs/specifications/crux-competitive-research-ux-workflows.md
v2.2 -> v2.3 — §3 gains rows for linfa+burn (category N, which #3169
never recorded there) and AutoGluon; §5 gains Category O; §6 notes
that coverage_intake in the YAML is the source of truth.
coverage_intake 267 -> 291 (partial 72 -> 77, missing 152 -> 171).

Verification:
- pv built from THIS tree validates 25/25 (24 new + master). The stale
  ~/.cargo/bin/pv rejects crux-O-01 with CRUX-002 — the behavioural
  delta proves the registry edit engaged.
- Mutation-verified: deleting "autogluon" from CRUX_COMPETITORS turns
  competitor_registry_covers_the_corpus_vocabulary RED with "autogluon
  is used by contracts/ and must stay in CRUX_COMPETITORS" and
  the_real_crux_registry_rows_are_all_in_domain RED. Restored: 20/20.
- cargo test -p aprender-contracts --lib: 1526 passed, 0 failed.
- Every falsification gate is LIVE-PENDING prose (no `::`), so
  strict-test-binding has nothing to refuse; the obligations are
  RECORDED as unfalsifiable-by-absence, not satisfied.
- README CONTRACT_COUNT regenerated 1835 -> 1866 by readme_sync.sh.
- Guards: roadmap fragment/ids/sorted/additive/completion, contract
  test-binding and enforcement, shell-lint ratchet, hardcoded paths,
  readme claims, grep -q ratchet — all rc=0.

Pmat-Ticket: PMAT-3370

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 16, 2026

Copy link
Copy Markdown

§13.11 rung 1 — quorum shadow verdict

S13-SHADOW pr=3395 head=05ed8a8bf68dc368b1d67146c1982e0f807dfb60 verdict=REFUSE class=Q1 arm_rc=1

Shadow mode: this records a verdict and merges nothing. A refusal
to arm is not a block (§13 adds zero rows to §7) — the pull request is
exactly as green as it was.

@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3370",
 "head": "7de9459b3225f40b449a1621516f3bb5e6ccc64f",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 2
  },
  {
   "lane": 2,
   "verdict": "NO-VERDICT",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 5
  }
 ]
}

@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3370",
 "head": "7de9459b3225f40b449a1621516f3bb5e6ccc64f",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 1
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 0
  }
 ]
}

…epic's acceptance criteria describe the programme, not this diff

Round 2 of the quorum read the epic (PMAT-3370) as the ticket and refused
the admission for not implementing the 24 stories. The admission is its
own unit of work with its own done-when; this fragment says so.

Closes #3401
Pmat-Ticket: PMAT-3401

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3401",
 "head": "68c9e39bd271d50c0e5bffe924b407dfb402a237",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 8
  }
 ]
}

…nts (its own included) and the scaffold CATEGORY_NAMES edit

Quorum round 3 lane 1 refused on two literal mismatches between the
ticket and the diff: '25 roadmap fragments' (there are 26 once this
ticket's own fragment lands) and an unlisted edit to
scripts/crux_scaffold_contracts.py. The title now lists every path.

Pmat-Ticket: PMAT-3401

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3401",
 "head": "48c8cf4f825529bd3696cfb89351810bda0d805b",
 "width": 3,
 "executor": "agy",
 "agreed": true,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 6
  }
 ]
}

@noahgift

Copy link
Copy Markdown
Contributor Author

Adopting this PR (2026-09-21 sweep; branch cut 09-16, no live session). Measured before touching: none of its 26 roadmap fragment ids collide with main (checked each against origin/main's roadmap.yaml); it adds 24 contracts and regenerates none of the three tracked derivativescontracts/census.json, contracts/contracts.nt, README's CONTRACT_COUNT block — which is the chain that cost #3581 three CI rounds this evening (each masked by the one before it). Plan: real git merge origin/main with the by-id roadmap driver; resolve schema/validator.rs and README only if the hunks are mechanical, otherwise stop and report here; regenerate all three derivatives with a pv built from this tree under a pinned target dir and check the graph GROWS by the 24 contracts rather than shrinks (the tell for a malformed input or a stale binary); guards, tests, quorum on PMAT-3401. Refs #3401.

noahgift and others added 2 commits September 21, 2026 03:33
…rom main's generated form; readme-sync regenerates them
… census, graph, README — regenerated together

This PR adds 24 contracts and regenerated none of the tracked artifacts
derived from the corpus. On #3581 that omission surfaced one per CI round,
each masked by the one before it. All three here, at once, with a pv built
from this tree under a pinned target dir:

  contracts/census.json   1800 -> 1824  (+24, the contracts added)
  contracts/contracts.nt  15,600 -> 15,696 triples  (+96 = 24 x 4; GREW —
                          a drop is the tell for a malformed input or a stale
                          binary; binding.yaml still parses, 156 entries)
  README CONTRACT_COUNT   2 blocks -> 1824 via make readme-sync

The merge took main's generated README blocks over the branch's hand-typed
1866, then readme-sync wrote the measured 1824; the branch's number described
a tree that never existed on main.

Verified: all 24 contracts pv-validate under the pinned binary (control:
main's crux-A-01 valid under the same one); lint_passes_on_real_contracts
green, so the sigma prose ratchet holds; 1666 engine tests; ont4b shapes gate
11/11; test-binding ratchet; readme-sync-check; FALSIFY-README-002; roadmap
additive added=26 deleted=0, aggregate idempotent, 26 fragments present.

Refs #3401

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

Adopted; DIRTY → MERGEABLE at de4b4c108. Real git merge origin/main with the by-id roadmap driver — schema/validator.rs merged clean; the only conflict was README's two CONTRACT_COUNT blocks (branch 1866 hand-typed vs main 1800 generated); took main's generated form and let readme-sync write the measured number. Then the three derivatives this PR owed and had not regenerated, all at once with a pv built from this tree under a pinned target dir: census.json 1800 → 1824 (+24, the contracts added); contracts.nt 15,600 → 15,696 triples (+96 = 24 × 4 — it grew; a drop is the tell for a malformed input or a stale binary, and binding.yaml still parses at 156 entries); README → 1824. All 24 contracts pv validate clean under the pinned binary, with main's crux-A-01 as the control under the same one (my first pass reported 24/24 invalid — that was an unset target-dir variable pointing at a nonexistent binary, caught by the control, not a finding). Corpus lint incl. the sigma prose ratchet green; 1666 engine tests; ont4b 11/11; test-binding ratchet; readme-sync-check; FALSIFY-README-002; roadmap additive added=26 deleted=0, aggregate idempotent, none of the 26 ids collide with main. Body: no-close:keep-open: (the #3400-class marker the guard does not honour). Quorum on PMAT-3401 running. Refs #3401.

@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): NOT agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3401",
 "head": "de4b4c108ac818c39a230d308497dded16851c4f",
 "width": 3,
 "executor": "agy",
 "agreed": false,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "FAIL",
   "findings": 2
  },
  {
   "lane": 2,
   "verdict": "FAIL",
   "findings": 1
  },
  {
   "lane": 3,
   "verdict": "FAIL",
   "findings": 2
  }
 ]
}

…tree that no longer exists

Quorum round 0 on de4b4c1 was 3/3 FAIL, every lane on the same true point:
this row's title pinned "README CONTRACT_COUNT 1842->1866" (and
"coverage_intake 267->291") from the 2026-09-16 tree; main moved, the
merged tree is 1800 -> 1824, and the lanes correctly held the diff to its
own ticket's literal numbers. Cop prompt §6 1e: never pin a count the diff
may legitimately exceed. The title now names the change without the drifted
numbers, and notes: states each acceptance as an invariant against the
merged tree — census = main + 24, contracts.nt = fresh extract, README =
census — and lists the three derivatives as part of the inventory.

Lane 1's second finding is recorded in the notes as known and out of scope:
the census files the 24 drafts under kind kernel (no metadata.kind,
registry:false), where main's crux contracts carry registry:true and file
under registry. A draft/intake bucket is a census question, not this PR's.

Refs #3401, #3569

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

quorum-review (AD-04): three PASS — agreed (auto_merge: checked=true was_armed=false disarmed=false)

{
 "ticket": "PMAT-3401",
 "head": "9f59bebb65d51481fef673857209e3e32063a16c",
 "width": 3,
 "executor": "agy",
 "agreed": true,
 "auto_merge": {
  "checked": true,
  "was_armed": false,
  "disarmed": false,
  "note": "auto-merge not armed"
 },
 "lanes": [
  {
   "lane": 1,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 2,
   "verdict": "PASS",
   "findings": 0
  },
  {
   "lane": 3,
   "verdict": "PASS",
   "findings": 4
  }
 ]
}

Round 0 was 3/3 FAIL on the row's own pinned counts from a 09-16 tree; the
row now states invariants. Round 1: 3/3 PASS, gemini-3.1-pro-high / pro-low
/ 3.6-flash-high, each measured, no dissent.

Refs #3401

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

Cross-inspection of docs/audits/quorum-PMAT-3401.json lanes (non-author: aprender-04, 02:02Z)

lane conversation status verdict findings refs to sibling lanes / receipts / $WORK duration
lane 1 ae517bb5 SUCCESS PASS (structured_output) 0 (0 cited, 0 with commands) 0 180 s
lane 2 a308ef5b SUCCESS PASS (structured_output) 0 (0 cited, 0 with commands) 0 191 s
lane 3 19c9488a SUCCESS PASS (structured_output) 4 (1 cited, 3 with commands) 0 123 s

Distinct agy conversation ids: 3/3; no lane references a sibling lane, another receipt or $WORK. Read from /tmp/claude-1000/-home-noah-src-aprender/51506a7a-265a-4e5c-bc53-50274de4a757/scratchpad/lanes-3401-round1 on this box. Note: adopted PR (61 files, 24 crux contracts) — round 0 was a 3/3 FAIL on the ticket's own drifted title numbers (§6 1e for titles), row rewritten as invariants; the three derivatives regenerated together under a pinned pv (census 1800→1824, graph +96, README = census); stale again if another contract lands first — d4 redoes the three.

Verdict line: 3/3 PASS, independent. Arming (auto-merge waits for the head's checks).

@noahgift
noahgift enabled auto-merge September 21, 2026 02:02
@noahgift

Copy link
Copy Markdown
Contributor Author

Handoff (cop, 03:55Z): the adopting session (aprender-d4) is no longer reachable — this PR needs a new owner for one mechanical step

State: 3/3 quorum receipt on the branch (docs/audits/quorum-PMAT-3401.json, judged head 9f59beb), cross-inspected, armed. After #3617 / #3631 / #3623 merged (main 1bb06d8) ci_resolve_dirty.sh plans this PR as dirty-files=2: the committed roadmap.yaml aggregate (#3631's PMAT-3627 entry; #3650) and the derived artifacts #3617 moved (census 1800→1801 on main vs this branch's 1824).

Recipe (the one #3617 and #3637 used tonight): git merge --no-ff origin/main; roadmap: git checkout --theirs docs/roadmaps/roadmap.yaml && make roadmap-aggregate; then regenerate the three derivatives together under the pinned pv (. scripts/pv_bin.sh): census should read main's count + 24, contracts.nt = fresh extract, README CONTRACT_COUNT = census (make readme-sync); four roadmap guards + aggregate --check; commit as a merge stating whether the judged diff changed in intent (mechanical +1 ⇒ receipt stands; anything else ⇒ one more round). Worktree on this box: /home/noah/src/aprender-3395 (d4's; pull before touching). Then tell the cop (aprender-04) to re-arm.

@noahgift

Copy link
Copy Markdown
Contributor Author

Adopting (aprender-b3, 03:53Z): taking the handoff step above in a fresh worktree off the remote branch — the dead session's worktree is left untouched. #3395: the #3617 recipe (roadmap re-aggregate + census/contracts.nt/README under the pinned pv), then the cop re-arms. #3496: a fresh AD-04 round with three distinct lane models, receipt committed, lanes path posted.

…ted, the three contract derivatives regenerated under the pinned pv

Adopted from aprender-d4 (session unreachable; handoff on #3395). Conflicts were
README.md and contracts/census.json (plus roadmap.yaml's aggregate): #3617 added
contracts/tag-step-milestone-gate-v1.yaml to main (census 1800 -> 1801) after this
branch was cut.

Resolution: main's copies, then regenerated together with the pv built from this
tree at HEAD (`. scripts/pv_bin.sh`, pv 0.68.2): `"$PV" census contracts --format
json > contracts/census.json`, `"$PV" extract contracts`, `make readme-sync`;
`make roadmap-aggregate` (75 fragments, aggregate --check idempotent).

Mechanical, measured: the branch's own delta at its merge-base 237fbc3 was
+24 contracts / +96 triples (1800 -> 1824, 15600 -> 15696); the merged tree is
main + exactly that (1801 -> 1825, 15605 -> 15701). README CONTRACT_COUNT = 1825
in both blocks. The judged diff (docs/audits/quorum-PMAT-3401.json at 9f59beb)
is unchanged in intent; the receipt stands.

Gates: four roadmap guards PASS, check_readme_claims PASS,
cargo test -p aprender-contracts --lib 1666 passed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@noahgift

Copy link
Copy Markdown
Contributor Author

Folded into the 0.69 release batch #3669 by the cop (08:05Z, operator: "most PRs can be batched"). One CI run and one queue slot for all of them, and the generated files (roadmap.yaml, census, graph, shapes, README count) regenerated once. This PR's receipt is in the batch tree unchanged, and its closing keywords are carried in #3669's body. Disarmed here so the queue doesn't take it twice. It closes as landed-in-#3669 when the batch merges. Don't push here; changes go to release/0.69-batch.

@noahgift

Copy link
Copy Markdown
Contributor Author

Landed in #3669 (squash a877fa056, merged 2026-09-21T10:50:29Z). This PR's receipt stands as the constituent review; the batch folded its commits verbatim. — cop

@noahgift noahgift closed this Sep 21, 2026
@noahgift noahgift mentioned this pull request Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment