Skip to content

Pre-deploy fixes for readiness contracts: the quiet monitor was off by one plateau; expectations keyed per page (browser v1.31.0, plugin v0.80.0) - #190

Merged
harper-joseph merged 3 commits into
mainfrom
fix/readiness-pre-deploy
Sep 19, 2026
Merged

harper-joseph merged 3 commits into
mainfrom
fix/readiness-pre-deploy

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

A pre-deployment review of what #187 shipped (browser 1.30.0 / plugin 0.79.0), before either reaches a node. Three reviewers read the browser path, the plugin path, and the deploy machinery independently; everything below was then reproduced against the compiled bits or against captured markup.

Report mode is confirmed non-gating and cheap (~0.5% of settle CPU; no settle phase changes; outcome never consults satisfied). What follows is what would have gone wrong after that — when the contracts are armed — plus what would have made the report window's data untrustworthy.

browser v1.31.0 — the DOM monitor did not measure quiet

onSatisfied: 'quiet' stops a render when the contract holds and the DOM has been quiet for quietMs. The second half comes from domMonitor.ts, and it had two defects:

quietMs() returned the start of the previous plateau. The scan found the last history entry outside tolerance and returned its timestamp — when the count was last far from its current value — rather than the next entry's, which is when the current plateau began. After a still period followed by a burst, it reported the pre-burst quiet as if it had accrued since. Reproduced against the compiled monitor: 2s still, 500 insertions, read 150ms later → 2,153ms quiet. On the canonical page shape (lull, then the rails arrive and make the last clause true) held && quiet stopped the instant the contract held, with no dwell — the stop-on-contract-alone behaviour we measured losing 99% of product links. It also explains why catalog needed quietMs: 750 where 250 lost links: the pre-rails lull was being misread as post-rails quiet.

The element count was inflated 2–3×. Every added node was countTree'd at callback time, so a parent inserted with its children in one task counted the children through the parent and again through their own records. Measured: parent + 10 children → +21; a 13,128-element document → 41,878, varying with where the parser yielded. Against a tolerance of 120, ~100 elements of carousel churn read as ~250 and the page read as never quiet — fail-safe, but every churning page would have paid the rot valve plus the full fallback settle.

Both fixed; test/domMonitor.test.ts pins them (there was no monitor test). Also: the monitor is not installed in report mode (nothing reads it, and it is the one page-visible artefact — a patched attachShadow); the observer stops polling once every clause has held (~half the evaluates per fleet render); the settle is wrapped so a throw releases the observer instead of letting the slow tail drop out of the satisfied_ms distribution.

plugin v0.80.0 — the expectation history mixed device variants

RenderExpectation was keyed by URL and fed by "the first stored variant" — desktop in steady state, but mobile whenever desktop failed to store. Measured on the live home page, same URL, one job: 625 img desktop vs 256 mobile, a 59% gap against a shortfall rule that fires at 50%. That artefact lands on the shortfall series and in the warn log looking exactly like real content loss — the one thing the table exists to detect.

Now keyed by cache key (URL + device), every stored variant recorded under its own key. 0.79.0 has not been deployed, so nothing migrates. Also: the table expires at 30 days and Target.delete cascades to it (it had no cleanup path); the pre-commit metric loop reads wire arrays defensively (a non-iterable from the unauthenticated result endpoint would have 500'd the result and held the lease); the missing-table warn is once-then-every-thousandth instead of once per render; verdict is documented as one per device variant (a two-device job emits two — read shares against render.time_ms, not render.outcome); learned: {} no longer wins over a sibling with observations. RenderQueue-level tests added for all of it.

console — the coverage guard has been red on main since 0.79.0

test/adminAssets.test.js correctly refuses a plugin metric no view reads, and render_readiness is one. Waived per series pointing at #189 (the Readiness panel); the metric is read raw from /prerender_admin/analytics during the report-only window.

Verification

  • browser 294/294, plugin 1172/1172, console 276/276; lint and format:check clean.
  • All five captured page types (PDP with/without reviews, populated/empty catalog, home) satisfy their contract under the shipped evaluator, and 9 of 9 simulated failure modes are caught (reviews missing, un-hydrated islands, empty rails, missing price, imageless tiles, missing hero…), using the contracts' own selectors.
  • assess() driven with the measured desktop/mobile counts against one URL-keyed row: 6 spurious shortfalls in 12 records, never converging. Against per-page rows: 0.

Deploy consequence

Stage 1.31.0 + 0.80.0, not 1.30.0 + 0.79.0. The fleet rollout is Recreate (all pods down before any come up), so shipping 1.30.0 and rolling again to fix the monitor before arming would be a wasted capacity-zero window.

🤖 Generated with Claude Code

harper-joseph and others added 2 commits September 18, 2026 23:20
… count was inflated; v1.31.0

Two defects in domMonitor.ts made `onSatisfied: 'quiet'` not do what it says.
Neither affects report mode; both had to be fixed before any contract is armed,
and the fleet has not yet deployed a version that carries the monitor.

1. `quietMs()` returned the timestamp of the LAST entry outside tolerance —
   the start of the plateau BEFORE the most recent change — instead of the entry
   after it, which is when the current plateau began. After a still period
   followed by a burst it reported the pre-burst quiet as if it had accrued
   since. Reproduced against the compiled monitor: 2s still, 500 insertions,
   read 150ms later → 2,153ms quiet. On the canonical page shape (lull, then
   the rails arrive and make the last clause true) "held && quiet" therefore
   stopped the instant the contract held, with no dwell — exactly the
   stop-on-contract-alone behaviour measured to lose 99% of product links.
   This is also why catalog needed quietMs: 750 where 250 lost links: the
   pre-rails lull was long enough to be misread as post-rails quiet.

2. The incremental element count ran `countTree` on every added node at
   callback time, so a parent inserted with its children in one task counted
   the children once via the parent and once each via their own records.
   Measured: parent + 10 children read as +21; a 13,128-element document read
   as 41,878, and the inflation depended on where the parser yielded. Against
   a tolerance of 120, ordinary carousel churn of ~100 elements read as ~250
   and the page read as never quiet — the fail-safe direction, but it would
   have sent every churning page to the rot valve and the full fallback settle.
   Fixed by skipping an added node whose ancestor (across shadow hosts) was
   added in the same batch.

Also in this change:
- The monitor is not installed in report mode. Nothing in that mode reads it,
  and it is the one page-visible thing a render leaves behind (a patched
  attachShadow whose source is not native). The patch keeps its `name` now.
- The observer stops polling once every clause has held: every later tick only
  overwrote a result the final look overwrites again. About half the evaluates
  on a fleet render.
- The settle is wrapped so the observer is released on a throw. Before, a
  render that failed mid-settle left it polling until the page closed and
  reported late or never, so the slow tail dropped out of the distribution the
  report window exists to measure.
- test/domMonitor.test.ts pins quiet-from-the-last-change, count-once for
  parent-then-children and for a parser-built document, tolerance semantics,
  the -1 answer on a truncated history, and open-shadow-root observation.
  None of these had a test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… variants are different pages; v0.80.0

A pre-deployment review of 0.79.0's readiness recording found that the
per-URL history was fed by "the first stored variant", which in steady state
is always desktop — but a result where desktop failed to store and mobile did
recorded MOBILE's counts against the desktop history. Measured on the live home
page in one job: 625 `img` on desktop, 256 on mobile, a 59% gap against a
shortfall rule that fires at 50%. On the `shortfall` series and in the log that
artefact is indistinguishable from real content loss — the one signal this
table exists to provide.

`RenderExpectation` is now keyed by cache key (url + device), exactly as the
page itself is, and every stored variant is recorded under its own key. 0.79.0
has not reached a node, so this is a schema change with nothing to migrate.

Also from the review:
- The table had no cleanup path. It now expires at 30 days (well past the
  longest render interval, and every governed render refreshes the row) and
  `Target.delete` drops the URL's rows beside `ProbeState`.
- The metric loop ran before anything was committed, outside any try/catch,
  and iterated wire-provided fields with `?? []` — a non-iterable from the
  unauthenticated result endpoint would have thrown the result away with a 500
  and held the lease. Arrays are read defensively now.
- A missing table (a node whose schema did not load) warned once per governed
  render — thousands an hour per node. Once, then every thousandth, with the
  running count.
- `verdict` is one per governed DEVICE VARIANT, not per render: a two-device
  job emits two. metrics.js and METRICS.md said "exactly one per render";
  shares should be read against `render.time_ms`, not `render.outcome`.
- `learned: {}` (a contract with no observe clauses) was truthy and could be
  picked over a sibling that had observations. `hasObservations` decides now.
- RenderQueue-level tests for all of the above; there were none.

console: the coverage guard in test/adminAssets.test.js fails on main since
0.79.0 — `render_readiness` is emitted and no view reads it. Waived, per
series, pointing at #189 (the Readiness panel). The metric is read raw from
/prerender_admin/analytics during the report-only window.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the readiness contract tracking to be per-page (URL and device variant) rather than per-URL, preventing false shortfall alarms caused by layout differences between desktop and mobile. It also optimizes the DOM monitor's element counting to avoid double-counting nested nodes, fixes the quiet period calculation, and ensures resources are cleaned up properly on failure. A review comment correctly identifies a potential bug in the DOM monitor's insideAdded helper, where standard HTML elements with a native string host property (like anchor or area tags) could cause incorrect traversal, and suggests a robust check to ensure host is only accessed when it is an object.

Comment on lines +99 to +102
const insideAdded = (node, added) => {
for (let p = node.parentNode || node.host; p; p = p.parentNode || p.host) if (added.has(p)) return true;
return false;
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The insideAdded helper traverses up the parent chain using node.parentNode || node.host. However, standard HTML elements like <a> (HTMLAnchorElement) and <area> (HTMLAreaElement) also have a native host property which returns a string (the hostname). If one of these elements is the root of an added subtree (having no parent), node.parentNode will be null, causing the traversal to fall back to node.host. This evaluates to a string, causing the loop to query added.has(string) and potentially leading to unexpected behavior or incorrect quiet measurements. To prevent this, ensure that host is only accessed if it is an object (representing a ShadowRoot host).

Suggested change
const insideAdded = (node, added) => {
for (let p = node.parentNode || node.host; p; p = p.parentNode || p.host) if (added.has(p)) return true;
return false;
};
const insideAdded = (node, added) => {
for (let p = node.parentNode || (node.host && typeof node.host === 'object' ? node.host : null); p; p = p.parentNode || (p.host && typeof p.host === 'object' ? p.host : null)) if (added.has(p)) return true;
return false;
};

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right — HTMLAnchorElement.host (and HTMLAreaElement's) is a string. As written it was benign (a string has no parentNode or host, so the walk ended on the next step) but it was walking through the wrong thing to get there. Fixed in e4dae55 the way the intent reads: follow host only from a ShadowRoot (nodeType === 11), via a small up() helper rather than the inline conditional. The parent-then-children test now also inserts an <a>-rooted subtree — a product tile is exactly that shape — and asserts it is counted once and agrees with recount().

…ncestor walk — <a> and <area> have a string one

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@harper-joseph
harper-joseph merged commit 0ca2a34 into main Sep 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant