Pre-deploy fixes for readiness contracts: the quiet monitor was off by one plateau; expectations keyed per page (browser v1.31.0, plugin v0.80.0) - #190
Conversation
… count was inflated; v1.31.0 Two defects in domMonitor.ts made `onSatisfied: 'quiet'` not do what it says. Neither affects report mode; both had to be fixed before any contract is armed, and the fleet has not yet deployed a version that carries the monitor. 1. `quietMs()` returned the timestamp of the LAST entry outside tolerance — the start of the plateau BEFORE the most recent change — instead of the entry after it, which is when the current plateau began. After a still period followed by a burst it reported the pre-burst quiet as if it had accrued since. Reproduced against the compiled monitor: 2s still, 500 insertions, read 150ms later → 2,153ms quiet. On the canonical page shape (lull, then the rails arrive and make the last clause true) "held && quiet" therefore stopped the instant the contract held, with no dwell — exactly the stop-on-contract-alone behaviour measured to lose 99% of product links. This is also why catalog needed quietMs: 750 where 250 lost links: the pre-rails lull was long enough to be misread as post-rails quiet. 2. The incremental element count ran `countTree` on every added node at callback time, so a parent inserted with its children in one task counted the children once via the parent and once each via their own records. Measured: parent + 10 children read as +21; a 13,128-element document read as 41,878, and the inflation depended on where the parser yielded. Against a tolerance of 120, ordinary carousel churn of ~100 elements read as ~250 and the page read as never quiet — the fail-safe direction, but it would have sent every churning page to the rot valve and the full fallback settle. Fixed by skipping an added node whose ancestor (across shadow hosts) was added in the same batch. Also in this change: - The monitor is not installed in report mode. Nothing in that mode reads it, and it is the one page-visible thing a render leaves behind (a patched attachShadow whose source is not native). The patch keeps its `name` now. - The observer stops polling once every clause has held: every later tick only overwrote a result the final look overwrites again. About half the evaluates on a fleet render. - The settle is wrapped so the observer is released on a throw. Before, a render that failed mid-settle left it polling until the page closed and reported late or never, so the slow tail dropped out of the distribution the report window exists to measure. - test/domMonitor.test.ts pins quiet-from-the-last-change, count-once for parent-then-children and for a parser-built document, tolerance semantics, the -1 answer on a truncated history, and open-shadow-root observation. None of these had a test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… variants are different pages; v0.80.0
A pre-deployment review of 0.79.0's readiness recording found that the
per-URL history was fed by "the first stored variant", which in steady state
is always desktop — but a result where desktop failed to store and mobile did
recorded MOBILE's counts against the desktop history. Measured on the live home
page in one job: 625 `img` on desktop, 256 on mobile, a 59% gap against a
shortfall rule that fires at 50%. On the `shortfall` series and in the log that
artefact is indistinguishable from real content loss — the one signal this
table exists to provide.
`RenderExpectation` is now keyed by cache key (url + device), exactly as the
page itself is, and every stored variant is recorded under its own key. 0.79.0
has not reached a node, so this is a schema change with nothing to migrate.
Also from the review:
- The table had no cleanup path. It now expires at 30 days (well past the
longest render interval, and every governed render refreshes the row) and
`Target.delete` drops the URL's rows beside `ProbeState`.
- The metric loop ran before anything was committed, outside any try/catch,
and iterated wire-provided fields with `?? []` — a non-iterable from the
unauthenticated result endpoint would have thrown the result away with a 500
and held the lease. Arrays are read defensively now.
- A missing table (a node whose schema did not load) warned once per governed
render — thousands an hour per node. Once, then every thousandth, with the
running count.
- `verdict` is one per governed DEVICE VARIANT, not per render: a two-device
job emits two. metrics.js and METRICS.md said "exactly one per render";
shares should be read against `render.time_ms`, not `render.outcome`.
- `learned: {}` (a contract with no observe clauses) was truthy and could be
picked over a sibling that had observations. `hasObservations` decides now.
- RenderQueue-level tests for all of the above; there were none.
console: the coverage guard in test/adminAssets.test.js fails on main since
0.79.0 — `render_readiness` is emitted and no view reads it. Waived, per
series, pointing at #189 (the Readiness panel). The metric is read raw from
/prerender_admin/analytics during the report-only window.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request refactors the readiness contract tracking to be per-page (URL and device variant) rather than per-URL, preventing false shortfall alarms caused by layout differences between desktop and mobile. It also optimizes the DOM monitor's element counting to avoid double-counting nested nodes, fixes the quiet period calculation, and ensures resources are cleaned up properly on failure. A review comment correctly identifies a potential bug in the DOM monitor's insideAdded helper, where standard HTML elements with a native string host property (like anchor or area tags) could cause incorrect traversal, and suggests a robust check to ensure host is only accessed when it is an object.
| const insideAdded = (node, added) => { | ||
| for (let p = node.parentNode || node.host; p; p = p.parentNode || p.host) if (added.has(p)) return true; | ||
| return false; | ||
| }; |
There was a problem hiding this comment.
The insideAdded helper traverses up the parent chain using node.parentNode || node.host. However, standard HTML elements like <a> (HTMLAnchorElement) and <area> (HTMLAreaElement) also have a native host property which returns a string (the hostname). If one of these elements is the root of an added subtree (having no parent), node.parentNode will be null, causing the traversal to fall back to node.host. This evaluates to a string, causing the loop to query added.has(string) and potentially leading to unexpected behavior or incorrect quiet measurements. To prevent this, ensure that host is only accessed if it is an object (representing a ShadowRoot host).
| const insideAdded = (node, added) => { | |
| for (let p = node.parentNode || node.host; p; p = p.parentNode || p.host) if (added.has(p)) return true; | |
| return false; | |
| }; | |
| const insideAdded = (node, added) => { | |
| for (let p = node.parentNode || (node.host && typeof node.host === 'object' ? node.host : null); p; p = p.parentNode || (p.host && typeof p.host === 'object' ? p.host : null)) if (added.has(p)) return true; | |
| return false; | |
| }; |
There was a problem hiding this comment.
Right — HTMLAnchorElement.host (and HTMLAreaElement's) is a string. As written it was benign (a string has no parentNode or host, so the walk ended on the next step) but it was walking through the wrong thing to get there. Fixed in e4dae55 the way the intent reads: follow host only from a ShadowRoot (nodeType === 11), via a small up() helper rather than the inline conditional. The parent-then-children test now also inserts an <a>-rooted subtree — a product tile is exactly that shape — and asserts it is counted once and agrees with recount().
…ncestor walk — <a> and <area> have a string one Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A pre-deployment review of what #187 shipped (browser 1.30.0 / plugin 0.79.0), before either reaches a node. Three reviewers read the browser path, the plugin path, and the deploy machinery independently; everything below was then reproduced against the compiled bits or against captured markup.
Report mode is confirmed non-gating and cheap (~0.5% of settle CPU; no settle phase changes;
outcomenever consultssatisfied). What follows is what would have gone wrong after that — when the contracts are armed — plus what would have made the report window's data untrustworthy.browser v1.31.0 — the DOM monitor did not measure quiet
onSatisfied: 'quiet'stops a render when the contract holds and the DOM has been quiet forquietMs. The second half comes fromdomMonitor.ts, and it had two defects:quietMs()returned the start of the previous plateau. The scan found the last history entry outside tolerance and returned its timestamp — when the count was last far from its current value — rather than the next entry's, which is when the current plateau began. After a still period followed by a burst, it reported the pre-burst quiet as if it had accrued since. Reproduced against the compiled monitor: 2s still, 500 insertions, read 150ms later → 2,153ms quiet. On the canonical page shape (lull, then the rails arrive and make the last clause true)held && quietstopped the instant the contract held, with no dwell — the stop-on-contract-alone behaviour we measured losing 99% of product links. It also explains why catalog neededquietMs: 750where 250 lost links: the pre-rails lull was being misread as post-rails quiet.The element count was inflated 2–3×. Every added node was
countTree'd at callback time, so a parent inserted with its children in one task counted the children through the parent and again through their own records. Measured: parent + 10 children → +21; a 13,128-element document → 41,878, varying with where the parser yielded. Against a tolerance of 120, ~100 elements of carousel churn read as ~250 and the page read as never quiet — fail-safe, but every churning page would have paid the rot valve plus the full fallback settle.Both fixed;
test/domMonitor.test.tspins them (there was no monitor test). Also: the monitor is not installed in report mode (nothing reads it, and it is the one page-visible artefact — a patchedattachShadow); the observer stops polling once every clause has held (~half the evaluates per fleet render); the settle is wrapped so a throw releases the observer instead of letting the slow tail drop out of thesatisfied_msdistribution.plugin v0.80.0 — the expectation history mixed device variants
RenderExpectationwas keyed by URL and fed by "the first stored variant" — desktop in steady state, but mobile whenever desktop failed to store. Measured on the live home page, same URL, one job: 625imgdesktop vs 256 mobile, a 59% gap against a shortfall rule that fires at 50%. That artefact lands on theshortfallseries and in the warn log looking exactly like real content loss — the one thing the table exists to detect.Now keyed by cache key (URL + device), every stored variant recorded under its own key. 0.79.0 has not been deployed, so nothing migrates. Also: the table expires at 30 days and
Target.deletecascades to it (it had no cleanup path); the pre-commit metric loop reads wire arrays defensively (a non-iterable from the unauthenticated result endpoint would have 500'd the result and held the lease); the missing-table warn is once-then-every-thousandth instead of once per render;verdictis documented as one per device variant (a two-device job emits two — read shares againstrender.time_ms, notrender.outcome);learned: {}no longer wins over a sibling with observations. RenderQueue-level tests added for all of it.console — the coverage guard has been red on
mainsince 0.79.0test/adminAssets.test.jscorrectly refuses a plugin metric no view reads, andrender_readinessis one. Waived per series pointing at #189 (the Readiness panel); the metric is read raw from/prerender_admin/analyticsduring the report-only window.Verification
format:checkclean.assess()driven with the measured desktop/mobile counts against one URL-keyed row: 6 spurious shortfalls in 12 records, never converging. Against per-page rows: 0.Deploy consequence
Stage 1.31.0 + 0.80.0, not 1.30.0 + 0.79.0. The fleet rollout is
Recreate(all pods down before any come up), so shipping 1.30.0 and rolling again to fix the monitor before arming would be a wasted capacity-zero window.🤖 Generated with Claude Code