Skip to content

Change probe files ONE url schedule row, not one per device (v0.70.0) - #172

Merged
harper-joseph merged 2 commits into
mainfrom
fix/probe-trigger-url-row-and-queue
Sep 17, 2026
Merged

harper-joseph merged 2 commits into
mainfrom
fix/probe-trigger-url-row-and-queue

Conversation

@harper-joseph

Copy link
Copy Markdown
Contributor

The bug

triggerRevalidate was missed by the v0.66.0 move to URL-keyed jobs. That change (#156) touched RenderQueue.js, Target.js and renderSchedule.js — but not changeProbe.js, which kept filing cacheKeysOf(url):

const keys = cacheKeysOf(row.url);            // url|desktop, url|mobile
await writeSchedules(keys.map((cacheKey) => ({ cacheKey, ... })));

And claim decides a job's devices from the row shape:

const device = CacheKey.deviceOf(granted.cacheKey);
const deviceTypes = device ? [device] : defaultDeviceTypes();

So every probe trigger produced two one-device jobs instead of one two-device job.

What that cost

  • Each job fetches the origin document for itself, which defeats the document reuse the browser gained in the same release (feat(browser): reuse the first device's document for the other devices of a job, and prefetch it ahead of the render (v1.24.0) #157). Probe-triggered renders were doubling origin document load — on a deployment that had explicitly asked for probe-driven origin load to come down.
  • Device drift. The two jobs are claimed and rendered at different times, so desktop and mobile land different lastCached — exactly the split-pair state URL-keyed jobs were built to remove.
  • Two schedule writes per trigger instead of one, on a path that runs in-line with the sweep, so it also lengthens every pass. On the deployment where this was found, the armed pass runs ~2.2× slower than the dry-run pass with bot traffic held constant (153,728/hr vs 150,743/hr across the two windows — contention ruled out by measurement), and trigger work is what's in the difference.

The fix

One schedule row, keyed by the URL. The pages are still expired per device — page content genuinely is per device; only the schedule collapses.

Every other writer already files the URL row (Target.put, Target.revalidate, and renderNow for a default device). A per-device row stays legitimate only for a deliberate one-device render, which is why claim keeps that branch. Rows written before this fix convert themselves the first time they render, so there's no migration and no sweep.

Tests

npm test in packages/plugin: 1050 pass, 0 fail. Lint and format:check clean.

New test pins the property directly — exactly one schedule row, keyed by the URL, with no | in the key, and both fromSitemap and effectiveInterval explicit (the funnel throws without them, since put replaces the record). The existing hard-expiry test now also captures schedule writes rather than only page patches, which is what let this hide.

Versioning

Tagged v0.70.0 to avoid colliding with the PRs already in flight — #162 and #163 both currently claim v0.67.0, with #164 at 0.68.0 and #165 at 0.69.0. If this should merge ahead of those (it is a live bug), renumber it to 0.67.0 and bump the others; the version bump is a one-line change in each.

Follow-up

The in-line trigger is the deeper issue: await trigger(row) runs inside the row handler, so trigger volume sets pass duration, which sets detection latency, which raises the change count per pass — a feedback loop no config knob escapes. Decoupling triggers onto a bounded, separately-paced queue is the next PR.

🤖 Generated with Claude Code

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates @harperfast/prerender to version 0.70.0 and resolves a regression from v0.66.0 where probe-triggered revalidations wrote per-device schedule rows instead of a single URL-keyed row. This fix prevents duplicate rendering jobs and ensures efficient document reuse. Feedback on the changes suggests removing a redundant mock and unused scheduled array from an existing test, as a new dedicated test has been added to explicitly verify the schedule write behavior.

Comment on lines +682 to +688
// Capture the schedule write too: the trigger must file ONE row, keyed by the URL.
const scheduled = [];
globalThis.databases.render_schedule.RenderSchedule = class extends FakeTable {
static async put(id, fields) {
scheduled.push({ id, ...fields });
}
};

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The scheduled array is defined and populated in this test, but there are no assertions verifying its contents at the end of the test. Since a new dedicated test ('a trip files ONE schedule row, keyed by the URL — not one per device') has been added to verify the schedule write behavior, this mock and the scheduled array are redundant here. Consider removing them to keep the test focused and clean.

@harper-joseph

Copy link
Copy Markdown
Contributor Author

Pushed a3c4e47: dropped the redundant scheduled capture from the hard-expiry test. It was mock setup carrying no assertion — the dedicated a trip files ONE schedule row, keyed by the URL test owns that behaviour. Good catch.

npm test in packages/plugin: 1050 pass, 0 fail.

harper-joseph and others added 2 commits September 17, 2026 17:37
… device; v0.70.0

`triggerRevalidate` was missed by the v0.66.0 move to URL-keyed jobs — that change
touched RenderQueue.js, Target.js and renderSchedule.js, but not changeProbe.js, so the
probe kept filing `cacheKeysOf(url)` (url|desktop, url|mobile) instead of the URL row.

`claim` gives a device-keyed row `deviceTypes: [thatDevice]` and a URL row the full
default set, so every probe trigger became TWO ONE-DEVICE JOBS instead of one two-device
job. Three costs, in order:

  - EACH JOB FETCHES THE ORIGIN DOCUMENT FOR ITSELF, which defeats the document reuse the
    browser gained in the same release. Probe-triggered renders were doubling origin
    document load — on a deployment that had asked for probe-driven origin load to come
    DOWN.
  - The two jobs are claimed and rendered at different times, so desktop and mobile land
    different `lastCached`: exactly the split-pair state URL-keyed jobs removed.
  - Two schedule writes per trigger instead of one, on a path that runs IN-LINE with the
    sweep, so it also lengthens every pass.

The PAGES are still expired per device — page content genuinely is per device. Only the
SCHEDULE collapses to one row.

Every other writer already files the URL row (`Target.put`, `Target.revalidate`, and
`renderNow` for a default device); a per-device row stays legitimate only for a deliberate
one-device render. Rows written before this fix convert themselves the first time they
render, so no migration.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review follow-up. The capture was added alongside the dedicated
'a trip files ONE schedule row, keyed by the URL' test and then never asserted on, so it
was mock setup carrying no property — the dedicated test owns that behaviour.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@harper-joseph
harper-joseph force-pushed the fix/probe-trigger-url-row-and-queue branch from a3c4e47 to 4cfafe4 Compare September 17, 2026 21:37
@harper-joseph
harper-joseph merged commit b82dc99 into main Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant