feat(accuracy): propagate end-to-end accuracy guarantees for approximate-over-approximate plans (#172, PR 1+2)

[zzylol]

Implements PR 1 (represent guarantees and fail closed) and PR 2 (conservative same-metric propagation + budget allocation) of the stacked plan in #172. PR 3 (TopK margin certificate, #239 posterior-bound integration) is deferred — CompositionOperator::TopKSelection exists and is always rejected by the default model, and GuaranteeSource::RuntimeObservation is reserved in the vocabulary, so PR 3 slots in without a schema change.

Refs #172.

What changed

asap-types — post_asap::guarantee (new)

• ErrorMetric (#[non_exhaustive]: AbsoluteValue, RelativeValue, Rank, Cardinality, Frequency, TopKMembership), each variant documenting its normalization — metrics are not interchangeable.
• BoundExpr / ProbabilityExpr: small serializable expression trees (Zero, Constant, Sum, Product, Scaled, Max, Unknown{statistic} / UnionBound, Scaled{count}), not prose. evaluate() returns None, never 0, when an Unknown leaf is reachable.
• GuaranteeSource provenance (exact, accuracy target, selected sketch params, child guarantee, composition step + rule name, budget allocation, unavailable statistic, runtime observation — reserved for Add posterior (query-time) error estimation for CMS/CountSketch, per Chen et al. IMC'21 #239).
• ResultGuarantee { metric, bound, failure_probability, provenance } with is_exact() / approximate_layer_count().
• CompositionOperator and typed AccuracyError::{UnsupportedComposition, MissingInputGuarantee, TargetNotSatisfied, NoLegalAllocation} (Serialize/Deserialize, thiserror).
• SummaryNode.guarantee: Option<ResultGuarantee> — Some on finalized values (SummaryEstimate readouts, ExactAggregate state, KeepPreAsap), None on raw sketch state and on families with no error model (unknown ≠ exact).
• SketchAlgorithm / SketchParams now derive Serialize/Deserialize (needed for structural provenance).

asap-aware-mapping — accuracy (new)

• AccuracyModel trait (local_guarantee / propagate / satisfies) and DefaultAccuracyModel:
  • local guarantees invert the existing sizing formulas (KLL 2/k, HLL 1.04/√2^p, KMV/Theta 1/√k, CMS-family e/w with δ = e^{-d}, DDSketch α);
  • rules: exact input (zero) → local; additive absolute B_in + B_out with union-bound δ; relative ε_in + ε_out + ε_in·ε_out (requires PropagationStats::values_non_negative == Some(true)); explicitly registered Lipschitz{L}; exact sum (Σ B_i, scaled by an unknown-unless-supplied row count); exact max/min (max B_i); everything else — including every cross-metric case and same-metric Rank/Cardinality/Frequency-over-itself — UnsupportedComposition. No independence is ever assumed; unknown statistics stay unknown.
• AccuracyBudgetAllocator trait + EqualSplitAllocator (ε_i = ε/n, δ_i = δ/n; (1+ε)^{1/n} − 1 for multiplicative metrics; no allocation for Exact).

asap-aware-mapping — replacement

• SketchAlgorithmStrategy::with_models(cost, accuracy, allocator); new/default_cost_model keep the defaults.
• construct_summary_agg computes the guarantee before the node exists, via propagate under the operator the family applies (ApproximateAggregate, ExactSum for Sum, ExactExtremum for MinMax, exact for Count, no finite Lipschitz constant for Rate/Increase). Approximate-over-approximate must also satisfies the outer node's own target; failure is ImplementError::Accuracy.
• New ReplacementStrategy::propose (default = replacements + no rejections). The sketch strategy proposes (a) the as-declared composition and (b) one candidate per allocator split, re-sizing the outer layer through CostModel::size_params and re-enumerating the child under its budget share (realize_child_with). Illegal attempts become RejectedCandidates on the new MemoGroup::rejected.
• search_workload_with_targets(roots_with_targets, strategies, accuracy_model) checks a root QueryRequirements.accuracy against the root group's bound candidates and moves misses to rejected before cost_sorted / global_selection run. A CostModel never sees a rejected candidate and cannot resurrect one.

DAG export (additive, backward compatible)

• SummaryDagNode.guarantee (omitted when None), guarantee under detail.guarantee in merged post-ASAP graphs, NamedGraph.rejections: Vec<TargetRejection> (omitted when empty); the dag_export devtool populates rejections by the same hash-then-structural-equality matching used for winners.

Precedence: root QueryRequirements.accuracy vs per-node AggIntent.accuracy

1. A root target passed to search_workload_with_targets is the end-to-end requirement for that query's root value; candidates whose guarantee is unknown or misses it are rejected before ranking. KeepPreAsap (exact) always survives, so an unsatisfiable root keeps the raw/pre-ASAP plan.
2. Approximate node over an exact child: the node's own AggIntent.accuracy sizes its sketch exactly as before; the readout's guarantee is that sketch's local guarantee. (Single-layer behavior is unchanged and not re-checked against its own target, so saturated/clamped sizings keep behaving as today.)
3. Approximate node over an approximate child: the outer node's AggIntent.accuracy is the end-to-end target for that value. The inner node's target is only a declared local requirement — the as-declared composition is kept only if the composed guarantee satisfies the outer target, and the allocator additionally proposes re-sized splits. Front-end-copied per-node targets are therefore never assumed to constitute a valid allocation.
4. AccuracyTarget::Exact admits only exact realizations; an allocation never overrides a child declared Exact.

Design decisions worth reviewing

• Guarantee lives on the value, not the state: sketch SummaryAgg nodes carry None; the SummaryEstimate carries the guarantee. Hydra grouping candidates copy the per-subpopulation guarantee (the shared-grid noise term from post-asap: GroupingStrategy axis - PerSubpopulationInstance vs SharedMultiSubpopulation (Hydra) summary types #256 is not folded in — flagged below).
• Honest failure probabilities for δ-blind sizings: KLL's k = 2/ε is attributed the 99%-confidence convention (δ = 0.01); HLL/KMV/Theta's standard-error sizing is attributed the 1σ level (δ ≈ 0.3173). This only bites when an EpsilonDelta target is checked end-to-end (root check or nested composition) — an Epsilon target is unaffected.
• TopK readouts carry a Frequency guarantee for each reported key's count plus an UnavailableStatistic{topk_membership_margin_certificate} provenance note; nothing claims TopKMembership until PR 3.
• Existing test updated: enumerating_the_targets_candidates_does_not_leak_into_a_nested_aggregate asserted the old unsafe behavior (KLL quantile composed over a KLL quantile with no check). It now injects a test-only permissive AccuracyModel so the property it is actually about still holds; a new test pins that the default rejects that composition.

Tests

cargo build --workspace, cargo test --workspace (all green, 165 in asap-aware-mapping), cargo clippy --workspace --all-targets -- -D warnings, cargo fmt --all -- --check.

From the issue's list: approximate-under-approximate rejected by default (not treated as exact, cross-metric and same-metric); exact child contributes zero error; additive bounds + delta union bound; relative error includes the cross term (and is rejected without sign knowledge); incompatible metrics rejected; Lipschitz; exact sum keeps an unknown row count unknown; equal budget allocation respects root ε/δ (additive and multiplicative); a candidate whose composed bound exceeds the target is absent before cost ranking and recorded as rejected; a legal tighter/more-expensive candidate (k=40) beats the illegal cheaper one (k=20) in global_selection; root target check removes candidates before ranking (Epsilon, Exact); DAG export contains metric, bound expr, failure probability, provenance, allocation and rejection reason. Deferred: TopK margin fixtures; shared-nested-summary costing (#172 cost-model section is out of this PR's scope).

Open questions for reviewers

• Are the attributed δ values for δ-blind sizings (KLL 0.01, HLL/KMV/Theta 0.3173) acceptable, or should those families' failure_probability be Unknown (which would make every EpsilonDelta end-to-end check fail for them)?
• The RelativeValue rule requires PropagationStats::values_non_negative == Some(true); the planner has no source for that statistic yet, so DDSketch-over-DDSketch is rejected by default until a deployment supplies it. Is that the right conservative stance for now?
• Hydra (SharedMultiSubpopulation) candidates inherit the per-subpopulation guarantee verbatim; folding Theorem 2's collision term in is left to post-asap: GroupingStrategy axis - PerSubpopulationInstance vs SharedMultiSubpopulation (Hydra) summary types #256/Support accuracy-aware approximate Count roll-ups #278 follow-up.
• search_workload_with_targets leaves logical Rewrite candidates (CSE/rollup/avg) untouched at the root check because they are not bound values; the targets inside a rewrite are their own groups. Worth confirming this matches the intended semantics for Support accuracy-aware approximate Count roll-ups #278.

🤖 Generated with Claude Code