Readiness contracts: stop a render when the page says it is complete, and report it when it is not

[harper-joseph]

Adds readiness contracts: a per-page-type statement of what a complete render contains, so the renderer stops when the page says it is done instead of when a timer expires — and so an incomplete render becomes a reported fact rather than a silence.

Spans both packages, because the browser produces the signal and the plugin is what makes it mean anything: @harperfast/prerender-browser v1.30.0, @harperfast/prerender v0.79.0. No-op unless configured; an unsatisfied contract always falls through to the existing settle, so a badly written contract can cost a render time but never content.

Why

Every settle signal we have is a timer, and a timer cannot be wrong out loud. A render that missed a widget, or that serialized pre-hydration markup, reports 200, non-empty and indexable, and nothing downstream can tell.

Measured under 6× CPU throttling: 8 of 15 renders finish incomplete and all 15 report outcome=ok — one storing 426KB instead of 958KB with 11 of 27 islands unhydrated and zero product links, while the existing waitFor review gate was satisfied. "A gate covering one widget is not a gate", measured.

Results

Live, five page types, paired A/B/A against a per-page churn floor measured from repeated baseline renders — counting content markers is not enough, since product-link and image counts move ±5% between two renders of the same code because the rails are personalised:

page	wall	CPU	parity	content
product	−43%	−25%	PASS	reviews 1430/1430, links 364/364
product (no reviews)	−47%	−35%	at floor	33/33, 352/352
listing	−34%	−28%	PASS (n=5)	links 658/658
listing (empty facet)	−54%	−33%	at floor	128/128
home	−31%	−14%	at floor	266/266

What is in it

Browser — six assertion forms, each because something else could not express it: presence, anyOf, absent, shed (frameworks drop a marker on hydrate), every+contains (relative, so no magic numbers), nonEmptyText. Any clause can carry onlyIf, keying it on what the page's own JSON-LD declares or what the DOM holds. A document-start MutationObserver answers "how long has this page been still?" in O(1), judged against the same tolerance the existing plateau uses.

Plugin — render_readiness, a new metric (verdict / unmet / shortfall / satisfied_ms), and RenderExpectation, a per-URL node-local record of what this URL last produced, written on the result path beside the probe claim.

The three properties the measurements forced

1. The quiet window is not optional, and it is where the speed comes from. Running the same stop policy with an empty contract saves the same time to within 30ms on 7 of 8 pages. Stopping the instant a contract holds saves more and loses the rails — 99% of product links on one product page, 100% on an empty facet — because rails have no server-rendered placeholder and no clause can assert one is still coming. The contract's contribution is falsifiability, which is what makes a short quiet window safe.
2. No verdict before the parser finishes, and a shed clause requires its elements to exist. Both were violated in development: a contract declared a page complete at 4ms and stored zero review nodes, because absence-shaped clauses are true on an empty document and it is quiet for the same reason.
3. Contracts rot, so they have to degrade. A clause that has never been true — once every other clause holds and the DOM is quiet — stands aside after unmetGraceMs, is reported unsatisfied, and the render falls back to the ordinary settle. Cost degrades to roughly today's behaviour rather than waiting out the timeout on every render forever; without that valve one unsatisfiable clause measured +385% wall. A slow page is a different failure and needs its own control — it is still mutating, so the valve does not fire — which is why timeoutMs should be capped low and tuned from the satisfied_ms distribution.

Learned expectations close the gap the contract cannot: a URL that carried three rails and 350 product links yesterday and none today has lost something no clause names. It converges rather than alarming forever — a shortfall is a vote, three in a row re-learns — and a suspected shortfall deliberately does not re-learn, because learning from a render we believe is short is how a real regression would erase its own evidence.

Also in here

• A latent bug fix. resolveConfigForJob cached resolved configs keyed by the identity of the overrides array, so two configs sharing that array resolved to whichever was seen first, for every URL matching an override. It presents as "this setting works on the home page and nowhere else" and it cost two wasted measurement runs. Now keyed on the config itself, with a regression test.

Review notes

• The cross-source guard is what I would most like a second opinion on. The naive any-of ("review items or a rating summary") is a measured false positive: the summary appears 236–263ms before the first review node on 6 of 13 renders, so it releases early on exactly the pages that have reviews.
• Every count pierces open shadow roots. A light-DOM query returns 0 on a page where the piercing walk returns 1,974.
• The expectation comparison exists in both packages (browser for the in-render path, plugin for detection). Same rule, and they must agree — both are commented to say so. I would take a suggestion for collapsing it.
• 284/284 browser tests, 1,164/1,164 plugin tests, lint and format clean. This repo has no PR CI, so that is a local verification rather than a gate.
• Version note: main is at plugin 0.78.0 and open Queue every detected change instead of deferring past a budget (v0.75.0) #178 bumps to 0.75.0, which would downgrade it. This takes 0.79.0.

Not in this PR

The per-page-type contract content is configuration and lives in the consumer — HarperFast/render-service#89, which is blocked on the v1.30.0 release.

🤖 Generated with Claude Code

---

Review provenance and what it changed

No independent cross-model coverage was available on this machine — agy is not installed, cursor-agent is missing, and the Codex leg failed on auth. What ran was an advisory same-family pass, which does not count as outside coverage, plus this repo's own CI reviewer. Stating that rather than implying the change was independently reviewed.

Seven defects were found and fixed before merge, none in the happy path:

found by	defect
CI reviewer	pollMs: 0 accepted — both the gate and the observer sleep on it, so zero is a tight loop calling into the page as fast as the event loop allows
CI reviewer	no ms field checked against 2147483647, past which setTimeout fires after 1ms — an over-large dwell silently becomes no dwell. scroll.topSettleMs is already guarded against this in the same module, so this was a consistency bug too
both	a malformed textMatches built its RegExp outside the evaluator's try, so one bad pattern threw out of the whole evaluation and discarded every other clause's result for that tick
advisory	readiness numerics unvalidated — a NaN timeoutMs makes the deadline NaN, the loop never runs, and the contract is silently disabled
advisory	stableMs documented on the contract and never read — dead config someone would set and believe. Removed rather than validated
advisory	report mode's "cannot change duration" was nearly true rather than true: stopping the observer waited out the rest of its poll interval. The sleep is now interruptible
advisory	the verdict metric double-counted on rebaseline, so the series stopped summing to one per render — which is how its own documentation frames it. Rebaselines now ride their own series

Validation coverage was widened past what was reported, because the same argument applies to every ms field: they all reach a timer or a comparison where a wrong value does nothing visible. They now go through one shared checker.

[gemini-code-assist]

Code Review

This pull request introduces 'Readiness contracts' to the browser rendering library, providing a mechanism to define page-type-specific assertions that determine when a render is complete. This replaces blind timer-based settling with a more reliable approach that combines content assertions and DOM quiescence monitoring. The changes include a new document-start monitor for tracking DOM stability and a history-based expectation system to detect content regressions. The review feedback correctly identifies the need for more robust configuration validation—specifically regarding timeout limits and regex patterns—and suggests implementing defensive error handling within the browser-side evaluation logic to prevent potential runtime crashes.

[gemini-code-assist]

The configuration fields like unmetGraceMs, quietMs, expectations.graceMs, contract.timeoutMs, contract.pollMs, contract.quietMs, and contract.stableMs are not validated in validateReadiness. In Node.js, passing a value greater than 2147483647 to setTimeout causes it to execute immediately (after 1ms), which can trigger unexpected hot loops or immediate timeouts. Additionally, if pollMs is zero or negative, it can cause an extremely fast, CPU-intensive loop in renderer.ts (line 716). We should validate these fields to be non-negative (or positive for pollMs) and not exceed 2147483647. Also, if textMatches is provided, we should validate it by trying to compile it with new RegExp(textMatches) to catch invalid regex patterns at config load time.

export function validateReadiness(readiness: unknown): void {
	if (readiness === undefined) return;
	const cfg = readiness as ReadinessConfig;
	if (typeof cfg !== 'object' || cfg === null) throw new Error('prerender config: readiness must be an object');
	if (cfg.onSatisfied !== undefined && cfg.onSatisfied !== 'quiet' && cfg.onSatisfied !== 'plateau') {
		throw new Error("prerender config: readiness.onSatisfied must be 'quiet' or 'plateau'");
	}
	const MAX_TIMER_MS = 2147483647;
	if (cfg.unmetGraceMs !== undefined && (typeof cfg.unmetGraceMs !== 'number' || cfg.unmetGraceMs < 0 || cfg.unmetGraceMs > MAX_TIMER_MS)) {
		throw new Error(`prerender config: readiness.unmetGraceMs must be a non-negative number up to ${MAX_TIMER_MS}`);
	}
	if (cfg.quietMs !== undefined && (typeof cfg.quietMs !== 'number' || cfg.quietMs < 0 || cfg.quietMs > MAX_TIMER_MS)) {
		throw new Error(`prerender config: readiness.quietMs must be a non-negative number up to ${MAX_TIMER_MS}`);
	}
	if (cfg.expectations !== undefined) {
		if (typeof cfg.expectations !== 'object' || cfg.expectations === null) {
			throw new Error('prerender config: readiness.expectations must be an object');
		}
		const exp = cfg.expectations;
		if (exp.graceMs !== undefined && (typeof exp.graceMs !== 'number' || exp.graceMs < 0 || exp.graceMs > MAX_TIMER_MS)) {
			throw new Error(`prerender config: readiness.expectations.graceMs must be a non-negative number up to ${MAX_TIMER_MS}`);
		}
		if (exp.tolerance !== undefined && (typeof exp.tolerance !== 'number' || exp.tolerance < 0 || exp.tolerance > 1)) {
			throw new Error('prerender config: readiness.expectations.tolerance must be a number between 0 and 1');
		}
		if (exp.rebaselineAfter !== undefined && (!Number.isInteger(exp.rebaselineAfter) || exp.rebaselineAfter < 1)) {
			throw new Error('prerender config: readiness.expectations.rebaselineAfter must be a positive integer');
		}
	}
	if (!Array.isArray(cfg.contracts)) throw new Error('prerender config: readiness.contracts must be an array');
	for (const contract of cfg.contracts) {
		if (!contract.name) throw new Error('prerender config: every readiness contract needs a name');
		if (!Array.isArray(contract.require) || contract.require.length === 0) {
			throw new Error(`prerender config: readiness contract "${contract.name}" needs at least one require assertion`);
		}
		if (contract.stableMs !== undefined && (typeof contract.stableMs !== 'number' || contract.stableMs < 0 || contract.stableMs > MAX_TIMER_MS)) {
			throw new Error(`prerender config: contract "${contract.name}" stableMs must be a non-negative number up to ${MAX_TIMER_MS}`);
		}
		if (contract.timeoutMs !== undefined && (typeof contract.timeoutMs !== 'number' || contract.timeoutMs < 0 || contract.timeoutMs > MAX_TIMER_MS)) {
			throw new Error(`prerender config: contract "${contract.name}" timeoutMs must be a non-negative number up to ${MAX_TIMER_MS}`);
		}
		if (contract.pollMs !== undefined && (typeof contract.pollMs !== 'number' || contract.pollMs <= 0 || contract.pollMs > MAX_TIMER_MS)) {
			throw new Error(`prerender config: contract "${contract.name}" pollMs must be a positive number up to ${MAX_TIMER_MS}`);
		}
		if (contract.quietMs !== undefined && (typeof contract.quietMs !== 'number' || contract.quietMs < 0 || contract.quietMs > MAX_TIMER_MS)) {
			throw new Error(`prerender config: contract "${contract.name}" quietMs must be a non-negative number up to ${MAX_TIMER_MS}`);
		}
		if (contract.pathPattern) {
			try {
				new RegExp(contract.pathPattern);
			} catch {
				throw new Error(`prerender config: readiness contract "${contract.name}" has an invalid pathPattern`);
			}
		}
		for (const assertion of [...contract.require, ...(contract.observe ?? [])]) {
			const a = assertion as Record<string, unknown>;
			const forms = [
				a.selector && !a.shed && !a.nonEmptyText,
				a.anyOf,
				a.absent,
				a.shed,
				a.every,
				a.nonEmptyText,
			].filter(Boolean).length;
			if (!assertion.name || forms !== 1) {
				throw new Error(
					`prerender config: readiness assertion in "${contract.name}" must have a name and exactly one of ` +
						'selector / anyOf / absent / (selector + shed) / (every + contains) / (selector + nonEmptyText)'
				);
			}
			if (a.textMatches !== undefined) {
				if (typeof a.textMatches !== 'string') {
					throw new Error(`prerender config: readiness assertion "${assertion.name}" has an invalid textMatches (must be a string)`);
				}
				try {
					new RegExp(a.textMatches);
				} catch {
					throw new Error(`prerender config: readiness assertion "${assertion.name}" has an invalid textMatches regex pattern`);
				}
			}
		}
	}
}

References

1. In Node.js, configuration options representing timeouts or delays passed to setTimeout must not exceed 2147483647 (the maximum 32-bit signed integer), as exceeding this limit causes Node.js to execute the callback immediately (after 1ms). Rather than silently clamping the value at the timer, validate and refuse invalid timeout values at configuration load time to prevent silent failures.

[harper-joseph]

Fixed in f2da097 — both parts of this were real, and one was worse than the comment suggests.

pollMs must be positive, not merely non-negative. I had allowed 0, and both the gate loop and the report-mode observer sleep on it, so zero is a tight loop calling into the page as fast as the event loop allows — a burned core per render. Now rejected.

The 2147483647 ceiling. You are right that this is the load-bearing one, and it is a consistency bug as much as a correctness one: scroll.topSettleMs is already guarded against exactly this in the same config module (a settle dwell past the timer ceiling is rejected — it would become NO dwell), and I did not follow the precedent I was sitting next to.

I widened the coverage past your list rather than field-by-field, because the same argument applies to every one of them: readiness.quietMs, readiness.unmetGraceMs, expectations.graceMs, expectations.tolerance (now bounded 0..1) and expectations.rebaselineAfter (positive integer) all reach a timer or a comparison where a wrong value does nothing visible. They now go through one shared checkMs helper so the next ms field added cannot be forgotten.

One correction to the suggested patch: it validates contract.stableMs, which no longer exists. It was documented on the contract type and never read by the gate — dead config that someone would set and believe — so it was removed rather than validated.

Tests added in readiness.test.ts for the ceiling, the zero poll, and each top-level/expectation field.

[gemini-code-assist]

If a.textMatches is an invalid regular expression string, it is not validated at config load time. Furthermore, in evaluateContract (which runs in the browser context), new RegExp(a.textMatches) is called outside of any try...catch block. An invalid regex will throw a SyntaxError synchronously, crashing the entire page evaluation loop. We should validate textMatches in validateReadiness and wrap the new RegExp instantiation in a try...catch block inside evaluateContract to fail gracefully.

		if (a.nonEmptyText === true && typeof a.selector === 'string') {
			let ok = false;
			let seen = 0;
			let pattern: RegExp | null = null;
			try {
				pattern = typeof a.textMatches === 'string' ? new RegExp(a.textMatches as string) : null;
			} catch {
				return { name: assertion.name, ok: false, count: 0 };
			}
			for (const root of roots) {
				try {
					for (const el of root.querySelectorAll(a.selector as string)) {
						seen++;
						const text = (el.textContent ?? '').trim();
						if (text.length > 0 && (!pattern || pattern.test(text))) ok = true;
					}
				} catch {
					return { name: assertion.name, ok: false, count: 0 };
				}
			}
			return { name: assertion.name, ok, count: seen };
		}

References

1. In browser-evaluated contexts (such as Puppeteer's page.evaluate), implement defensive checks (e.g., Array.isArray guards, try/catch blocks) even if they are redundant with upstream validation. A runtime exception inside page.evaluate can fail the entire render job, so it is critical to handle errors gracefully and skip malformed rules rather than throwing.

[harper-joseph]

Fixed in b3d06c6, and your reasoning about why it matters in a page.evaluate context is the part I would highlight for anyone reading this later.

The RegExp was built outside the loop’s try, so a malformed textMatches threw out of the whole evaluator — discarding every other clause’s result for that tick, not just its own. Since the evaluator is the thing deciding whether the page is complete, that failure mode is worse than a crash: the contract would report every clause unmet and the render would look incomplete for a reason that has nothing to do with the page.

Both halves are now in place, as you suggested: compiled inside a try so a bad pattern fails only its own clause, and validated at config load (f2da097) so it cannot reach a render at all. The in-evaluator guard is kept deliberately as defence in depth — it is unreachable through the public API now, and the comment says so, because the alternative shape is the one that just bit.