Surface raw-document caching, sitemap conditional fetching and explain cadence in the console (v0.14.0)

[harper-joseph]

Console catch-up with plugin v0.77.0 (#165, #177, #181, #182). Four signals the plugin now emits or returns had no console surface, and each is invisible by construction rather than merely missing.

@harperfast/prerender-console 0.13.1 → 0.14.0 — new panels and a relabelled field, no breaking change. No console version is reserved in an open PR (#178 is plugin v0.75.0) and prerender-console-v0.14.0 is unreleased.

1. The raw-document cache reports what it REFUSED (#181)

New Traffic panel over prerender_ops series raw_cache. It leads with the refusals, because that is the whole point: a route that is enabled and storing nothing produces the same miss rate, the same origin proxies and the same absence of errors as a route nobody enabled. Charting stored alone would have been blind to the one failure mode the series exists for.

Two outcomes are findings rather than traffic and are called out above the breakdown:

• has-cookie — the origin set a cookie, i.e. it is personalizing a route enabled on the assumption it is shared. The refusal is correct; the assumption is not, and nothing else in this console would ever say so.
• oversize — render.raw.maxBytes sits below the route's real document size, so the feature is on and structurally cannot fill. A settings fix.

The rest (not-200, content-type, no-store, no-body, staging, capture-busy, capture-failed, write-failed, empty) get a row each with what they mean. Off and never configured, the panel describes the capability instead of drawing a chart of zeroes — the same exit the discovery gate takes, and it sits next to it: the gate keeps those URLs out of the rotation, the raw cache answers them anyway.

Store rate is deliberately not divided by serves. A document stored now is read by the next crawler, possibly in the next window. Different populations, shown side by side.

2. raw as a serve verdict, made coherent everywhere else

The colour entries in charts.js were already in place. This is what consuming them correctly turned out to require:

• The non-hit taxonomy gains a raw family. Without it the verdict fell through to "an outcome this console does not know about" — a cache serve that costs the origin nothing, filed under unknown.
• The Cache-served tile now counts the VERDICT, not the source. It counted path === 'cache', which agreed with isCacheServed until v0.76.0 because every cache-served verdict carried source cache. raw breaks that — it is a cache serve with its own source — so the tile would have fallen as the feature started working, while gross offload rose. It now uses isCacheServed (the enumeration charts.js says every such sum must share) and names the raw share in the subtitle rather than folding it in: a snapshot means a render covers that URL, a raw document means one never will.
• The staleness panel's past-due denominator moves the other way, to source cache only. page_age/route_page_age are emitted only when the source is cache, so a raw serve contributes no age sample; counting it in that denominator would shrink the past-due share on exactly the deployments serving a lot of raw, and fire the "the verdicts disagree with this ratio" note against a fleet that is genuinely behind.

That last pair is the answer to "is the raw addition coherent with how these sets are consumed?" — it was not, in two opposite directions, and one closed set (NOT_HIT) had no entry at all.

3. Sitemaps: ingested vs checked, and the walk counters (#165, #177)

lastRefreshed was mislabelled and is now "ingested". A 304 deliberately writes nothing — the stored row and its validator are still current — so that field is when this document's entries last changed, not when it was looked at. Labelled "refreshed", conditional fetching working read as a sitemap hours stale. When it was checked comes from the run row (SitemapRefresh, one per root) and is shown beside it, on the root cards and in the detail stats. A child sitemap has no run row, and the card says which timestamp it is carrying.

The per-root last walk now reports Not modified (304) — N of M documents and, on the creates line, how many took the new-target fast path — a subset of created, stated with its denominator so it can never read as a fourth count.

New walk-activity panel summing the sitemap_* counters across roots and nodes over 24h (the range key the Change probe view already uses, so the two share one cached scan per node). sitemap_not_modified non-zero is the only evidence anywhere that conditional fetching is working at all — a walk that re-parses everything succeeds exactly like one that skipped — so zero 304s across a day of walks is flagged explicitly.

4. Explain returns a cadence block, and the console renders it (#182)

The explainer's Target card showed renderInterval, which is the ceiling the demand ladder schedules inside; with the ladder armed that is not the cadence for most of a corpus, and nothing said so. A new Render cadence card renders the plugin's own resolution — effective interval, all four inputs, the demand floor — and leads with clampedBy:

• floor — the ladder wanted faster and the floor refused. Across a route it means the ladder has no dynamic range there and promotion is running for nothing.
• ceiling — the rung is slower than the route grants, so it is inert.

The Target row is relabelled Render interval (ceiling). A URL with no target gets no card: cadence is a property of being in the rotation, and absence is the answer rather than a row of dashes.

The metric guard's sitemap_* blind spot is closed

metrics.sitemapRun is on DYNAMIC_SERIES_SLOT, and that exemption cost exactly what the probe one did: sitemap_not_modified (v0.69.0) and sitemap_created_soon (v0.74.0) shipped unread with a green suite on both sides.

The new test reads the emit sites in resources/Sitemap.js, deliberately not the catalog as the probe test does. The two have drifted and the drift is the point: the catalog's values list enumerates six sitemap_* names while the plugin emits fifteen, so a catalog-based guard would have passed while blind to the two series it exists for. Verified to bite (renaming a read makes it fail with the series name).

What I did NOT fix, explicitly

• sitemap_departure_* (ten series) still has no panel. It is now named in the guard with the reason rather than passing silently: departures are a separate decision surface with dry-run-vs-armed semantics of their own and want their own panel, not seven more tiles on the walk card. departure_reattached is the one METRICS.md says to watch.
• The plugin's METRICS.prerender_ops.dimensions.path.values list is stale — it is missing raw_cache, sitemap_not_modified, sitemap_created_soon and the whole departure_* family. That is a plugin-side catalog fix and would need a plugin version bump, so it is out of scope for a console PR. The new guard routes around it by reading emit sites; the probe guard still reads the catalog and is therefore exposed to the same drift.
• serve_error gained raw-blob-timeout / raw-blob-unreadable details. serve_error is waived in the coverage guard (the serve-side view of it is bot_serve blob-* on Traffic) and those two details have no serve-side equivalent, since a raw blob fault degrades to an origin proxy counted as miss. Left as-is; worth a follow-up if raw caching goes wide.

Verification

• packages/console: 276 tests pass (was 261; the guard was already failing on raw_cache at origin/main). New: 4 raw-cache Traffic tests, 5 sitemap ingest/walk tests, a new inspectView.test.js with 5 cadence tests, and the sitemap-family guard.
• packages/plugin: 1155 tests pass (untouched, run as a sanity check).
• npm run lint and npm run format:check clean at the root.

🤖 Generated with Claude Code

[gemini-code-assist]

Warning

Gemini encountered an error creating the review. You can try again by commenting /gemini review.