feat(dag-viewer): export and visualize planner cost/benefit annotations (#286)

[zzylol]

Closes #286.

Update: review fixes (latest commit)

A code review found 5 confirmed bugs, all fixed on this branch:

1. Single-query "Workload cost" panel was dead on the primary UI path.
   viewer.js's loadFiles()/loadWorkload() constructed each query
   object with a field allowlist that omitted the new workload_cost
   field — the headline feature never rendered through drag-and-drop, the
   file picker, or the planner UI, even though the exported JSON had the
   data. Both loaders now forward workload_cost.
2. Cross-file decision.id collision could silently undercount a
   multi-query total. decision.id is only unique within one
   dag_export invocation, not across independently loaded files.
   computeSelectionWorkloadCost now dedups by ${sourceBatch}: ${decision.id}, where sourceBatch is a viewer-assigned id shared by
   every query loaded from the same document.
3. Edge occurrence count conflated with distinct consumer count.
   shared_node_edge_annotations counted Vec occurrences into a shared
   child as consumer_count; a Join whose left and right operands are
   the same Rc (post pointer-dedup) inflated that to 2 for one real
   consumer, halving the reported per-edge cost and producing two
   colliding EdgeCostAnnotation entries that viewer.js's
   edgeCostByPair Map then silently overwrote. Switched to a HashSet
   of distinct parent ids; added a regression test.
4. total_cost() validated horizon but not the rate/one-shot inputs
   themselves. A non-finite recurring_cost_rate (e.g. a stray NaN)
   silently produced Some(NaN) instead of following the module's own
   "never fabricate, never a poisoned total" rule. Both inputs are now
   validated finite.
5. Doc/implementation mismatch on NamedGraph.workload_cost's dedup
   scope. The doc claimed cross-query dedup; the implementation only
   ever dedups within one query (WorkloadGraph.workload_cost is the one
   that dedups across all queries). The implementation was already
   correct — only the doc was wrong, and is now fixed to say so, pointing
   readers at WorkloadGraph.workload_cost for cross-query totals.

Also addressed two of the three lower-priority follow-ups: cost: f64 is
now derived from selected_cost.value at both call sites instead of being
set independently (closing the "kept in sync by convention" gap), and
default_cse_recompute_cost is now memoized once per winner instead of
re-walking the same subtree on every matched node position (and
shared_node_edge_annotations's parents_of map is now built inline in
deduplicate_pointer_shared_nodes's existing loop instead of a second
full pass). Not fixed: computeSelectionWorkloadCost in viewer.js still
hand-reimplements cost.rs's dedup-and-sum algorithm in JS with no shared
source of truth — there's no JS/Rust code-sharing mechanism in this tool,
so this remains a manual-sync follow-up.

Verified with cargo build --workspace, cargo test --workspace (all
green), cargo clippy --workspace --all-targets -- -D warnings (clean),
cargo fmt --all -- --check (clean), and
python3 -m unittest discover -s tools/dag-viewer (18/18). node --check
remains unavailable in this sandbox; the JS fixes were verified by careful
manual review instead (see "Testing" below for the same caveat on the
original diff).

Summary

Adds a structured, optional planner/exporter cost-and-benefit annotation
(asap_types::cost::CostAnnotation) and wires it through dag_export's
JSON output and tools/dag-viewer's sidebar/on-graph UI.

Rust: crates/types/src/cost.rs (new)

• CostAnnotation { value, unit, source, baseline, delta, benefit_ratio, model_version, benchmark_id, inputs } — matches the issue's sketch, plus
  an explicit benefit_ratio field (kept separate from delta so a
  consumer never has to recompute it from delta/baseline).
• CostUnit: CostUnitsPerSecond (the issue's rate formulas),
  CostUnits (a total_cost(H) one-shot/finite-run total), and
  RelativeStructuralUnits (a dimensionless structural-size proxy — see
  "Deferred" below).
• CostSource: Modeled / Measured / Unavailable. A missing value is
  always Unavailable with value: None — never 0 or another synthetic
  number.
• BaselineRef: PreAsapRecomputation, HighestRankedNonSelectedCandidate { rank }, Named(String) — the issue's own two named examples, as
  first-class variants.
• total_cost(recurring_rate, horizon, one_shot) — rate * H + one_shot,
  requires an explicit, finite, non-negative horizon for any recurring
  term; rate and one-shot are separate arguments so they can never be
  silently added by a caller before reaching this function.
• sum_workload_costs / workload_cost_summary — sum a workload's
  per-decision cost annotations, deduplicating by an explicit key (so a
  node/decision shared across queries/regions is counted once), and
  rejecting (not silently mixing) unit-mismatched aggregation.
• 11 unit tests, including explicit double-counting regression tests
  (sum_workload_costs_counts_a_shared_node_once,
  workload_cost_summary_computes_benefit_from_deduplicated_totals).

Rust: crates/types/src/dag_export.rs

• DagGraph gains edge_annotations: Vec<EdgeCostAnnotation> ({from, to, cost}), populated by deduplicate_pointer_shared_nodes for every edge
  running into a genuine DAG merge point — a node id referenced by more
  than one parent after export_post_asap's own dedup pass (real Rc
  sharing, not necessarily a SharedSubtreeStrategy decision). This is
  scoped exactly to the issue's "edge cost only when genuinely attributable
  to the edge" bullet; arbitrary multi-hop path cost is explicitly not
  attempted, matching the issue's own scope note.
• DagDecision and TargetReplacement gain baseline_cost /
  selected_cost / benefit: Option<CostAnnotation>, additive alongside
  their existing bare cost: f64 (unchanged — full backward compat, see
  below).
• NamedGraph / WorkloadGraph gain workload_cost: Option<WorkloadCostSummary> (whole selected-workload baseline/selected/
  benefit).
• 2 new tests confirming export() (plain, tree-only) never produces edge
  annotations, and export_post_asap() does for a real shared Rc — plus
  an updated existing test asserting edge_annotations is omitted (not
  []) when empty.

Rust: crates/devtools/src/bin/dag_export.rs

• winner_cost_annotations(target, consumer_count, selected_cost) computes
  baseline/selected/benefit for one winning candidate:
  • baseline = default_cse_recompute_cost(target) * consumer_count
    (BaselineRef::PreAsapRecomputation) — the same structural-size
    function asap_aware_mapping::cost_model::DefaultCostModel already
    uses, not a second formula. Always computable (never Unavailable).
  • selected = the winning candidate's own RankedGroup::costs[0]
    (already computed for ranking); Unavailable exactly when that's
    NaN (the cost model has no estimate for that candidate shape).
  • benefit = baseline - selected, with benefit_ratio guarded at
    baseline <= 0.
  • For a winning SharedSubtreeStrategy/CseShare decision this is
    literally "avoided recomputation for a shared sub-DAG" (issue
    granularity item feat(core): interface types for L1 input, L3 intent algebra, L4 sketch algebra #3) — no separate mechanism needed, since the
    baseline is exactly the cost of recomputing independently.
• Wired into both additive outputs (TargetReplacement, and every node's
  DagDecision inside post_graph), so they never disagree.
• Whole-workload totals (NamedGraph.workload_cost /
  WorkloadGraph.workload_cost) are built by deduplicating every decision
  in scope by decision.id — already a collision-free key (one winner
  index), reused directly as sum_workload_costs's dedup key rather than
  inventing a second identity.
• tools/dag-viewer/dag.example.json regenerated via
  generate-sample.sh → real lowering → ASAP-aware mapping → post-ASAP →
  dag_export, not hand-patched. (One honest data point worth flagging:
  the regenerated workload_cost.benefit for that sample workload is
  negative — today's structural-size proxy isn't always favorable to the
  selected strategy. That's real output, not a bug; see "Deferred" below.)

JS: tools/dag-viewer/viewer.js / index.html

• Concise on-graph ▼NN% / ▲NN% badge appended to a post-ASAP node's
  label when its decision has a benefit value (▼ = cheaper than
  baseline, ▲ = more expensive) — nothing appended when unavailable.
• Sidebar (renderCostAnnotation/renderDecisionCostBlock): full
  value/unit/Modeled-Measured-Unavailable badge/baseline/delta/ratio/
  model-or-benchmark-version/inputs breakdown, on node click (replacement
  decision cards) and edge click (EdgeCostAnnotation).
• Workload-scope cost summary in the scope picker: single-query selection
  reads the exporter's own precomputed NamedGraph.workload_cost directly;
  multi-query selection aggregates the already-exported per-decision
  annotations client-side, deduplicated by the explicit decision.id
  field (mirrors the Rust-side algorithm exactly) — this is aggregation of
  explicit JSON fields, not client-side cost estimation, and it refuses to
  aggregate mismatched units rather than mixing them.
• All rendering is null-safe: an older export with none of these fields
  renders exactly as before.

Deliberately deferred / stubbed (and why)

• Real cost-per-second rates (CostUnitsPerSecond): today's cost model
  (asap_aware_mapping::cost_model) has no update_rate /
  evaluation_rate / query_interval inputs at all — that's Recurrence-aware optimization: cost shared maintenance by query repetition #287's job.
  Every annotation this PR produces is honestly unit-tagged
  RelativeStructuralUnits instead of mislabeling a structural-size proxy
  as a real rate. The CostAnnotation/CostUnit plumbing already accepts
  CostUnitsPerSecond unchanged, so wiring in Recurrence-aware optimization: cost shared maintenance by query repetition #287's inputs later is a
  producer-side change only.
• Measured annotations (CostSource::Measured, benchmark_id): the
  schema supports this end-to-end (JS renders it with a distinct badge
  color), but nothing populates it — Evaluate AHA-style sparse-subpopulation strategy vs. maintaining full hierarchical sub-population summaries #288's benchmark artifact/contract
  doesn't exist yet, per the issue's own scope note.
• Ordinary (non-decision) node costs: only nodes carrying a decision
  (a winning replacement) get a cost annotation. Plain IR nodes (Scan,
  Filter, Join, ...) have no cost-model hook to estimate from today, so no
  annotation is attached at all — no Unavailable placeholder is invented
  where nothing was ever asked for.
• Edge annotations in union/workload-collapse mode: edge_annotations
  is only wired into the single-query lane view. Attribution across
  collapsed cross-query union nodes is genuinely ambiguous (which query's
  edge "owns" a cost after nodes from different queries have been merged
  into one visual node) and didn't seem worth guessing at.

Design decisions where the issue was ambiguous

• Kept DagDecision.cost: f64 / TargetReplacement.cost: f64 unchanged
  and additive rather than replacing them with the new structured type —
  the issue's own acceptance criteria requires "Existing exports without
  annotations remain fully supported."
• benefit_ratio is its own explicit field on CostAnnotation rather than
  only delta (as the issue's literal sketch has it) — the issue's own
  acceptance criteria requires the ratio to be renderable, and recomputing
  it from delta/baseline on every consumer seemed worse than computing
  it once, correctly (with the baseline <= 0 guard), at the source.
• Chose default_cse_recompute_cost(target) * consumer_count as the
  default "pre-ASAP recomputation" baseline (rather than "highest-ranked
  legal non-selected candidate") since it's always computable (a target
  with only one candidate still gets a meaningful baseline) and it's the
  more natural "why is this beneficial at all" comparison the issue's own
  context section asks for.

Testing

• cargo build --workspace — clean.
• cargo test --workspace — all green (146 in asap-types, 6 in the
  dag_export binary, no regressions elsewhere).
• python3 -m unittest discover -s tools/dag-viewer -p test_render.py —
  18/18 pass.
• node --check unavailable in this environment (no Node.js installed);
  substituted careful manual review of every JS diff plus end-to-end
  smoke-testing (dag_export → JSON → render.py → inlined HTML,
  inspected the resulting JSON for shape/values by hand) and the existing
  Python test suite, which inlines and structurally checks viewer.js. I'd
  flag re-running node --check tools/dag-viewer/viewer.js (and eyeballing
  the page in a real browser) as a good first thing for a human reviewer
  with Node available to do.

Open questions / follow-ups for a reviewer

• Whether RelativeStructuralUnits values should be hidden from the
  on-graph percentage badge until real rate/measured data exists (right
  now a badge can read e.g. ▲25% off a structural proxy, which is
  directionally informative but not a real cost-per-second claim) — I left
  it visible since the sidebar always makes the unit/provenance explicit
  on click, but a reviewer more familiar with how this tool gets used
  might prefer to gate the on-graph badge to Modeled-with-a-real-rate-
  unit or Measured only.
• Recurrence-aware optimization: cost shared maintenance by query repetition #287 (recurrence-aware modeled inputs) and Evaluate AHA-style sparse-subpopulation strategy vs. maintaining full hierarchical sub-population summaries #288 (measured sparse-vs-
  maintained annotations) are the natural next steps to move
  RelativeStructuralUnits output toward real CostUnitsPerSecond /
  Measured annotations; no plumbing changes should be needed on this
  side when they land.

🤖 Generated with Claude Code