Skip to content

feat(accuracy): propagate end-to-end guarantees for nested summaries (#172) - #303

Merged
zzylol merged 12 commits into
mainfrom
review-pr-299-work
Aug 28, 2026
Merged

feat(accuracy): propagate end-to-end guarantees for nested summaries (#172)#303
zzylol merged 12 commits into
mainfrom
review-pr-299-work

Conversation

@zzylol

@zzylol zzylol commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Why

Approximate summaries need one caller-visible proof. Sizing each sketch independently can exceed the requested error budget, mix incompatible metrics, or let cost ranking select an unprovable result. This applies to single and nested summaries. Refs #172.

What

  • Adds typed, serializable result guarantees with symbolic bounds, failure probabilities, provenance, and estimator-contract identity.
  • Allocates accuracy budgets and filters candidates against AccuracyTarget before costing.
  • Propagates supported exact, additive, relative, Lipschitz, sum, and extremum guarantees.
  • Models distinct contracts for CMS L1 frequency and CountSketch L2 frequency.
  • Wires typed planning evidence into TopK membership-margin certificates and Hydra shared-grid composition.
  • Keeps HLL failure probability unknown because its current parameters encode precision, not a confidence-level budget.
  • Exports guarantees, allocations, and rejected-candidate reasons.

How

AccuracyModel is the correctness boundary; AccuracyBudgetAllocator proposes per-layer budgets; AccuracyEvidenceProvider supplies typed planning-time observations; CostModel sees only candidates that pass propagation and target checking. NoAccuracyEvidence is the default, so unknown statistics and unsupported metric/operator combinations fail closed.

Key built-in contracts include:

  • KLL single-sided 99th-percentile rank error: 2.296 / k^0.9723, inverted for sizing and identified with a pinned Apache DataSketches contract.
  • HLL: 1.04 / sqrt(2^p) RSE magnitude with unknown failure probability; it can satisfy Epsilon, but not EpsilonDelta.
  • KMV/Theta: conservative 99% Chebyshev contracts from their stated estimator variance.
  • CountSketch: sqrt(3 / width) * ||f||_2 with odd median depth and a Hoeffding failure bound.
  • Hydra: B_inner + B_shared_grid, with failure probabilities union-bounded.

Before this PR

outer approximate operator
└─ inner approximate estimate

Each layer could be sized locally, but the finalized plan had no machine-readable end-to-end proof. TopK and Hydra formulas had no production evidence path, and HLL incorrectly converted its RSE into a claimed 99% confidence bound despite having no confidence parameter.

After this PR

candidate
  -> allocate budget
  -> derive local guarantees
  -> consume typed evidence when required
  -> compose end-to-end guarantee
  -> check AccuracyTarget
  -> cost legal candidates only

A default ε=0.01 KLL quantile is sized to k=269 under the documented single-sided 99% fit. TopK and Hydra can consume supplied evidence through candidate construction; absent evidence retains the exact/pre-ASAP fallback. Generic HLL no longer claims a failure probability that its parameters cannot prove.

Design decisions

  • Keep AccuracyTarget authoritative but extensible; a request and its proof are separate types.
  • Preserve rank, cardinality, value, L1-frequency, L2-frequency, and membership metrics instead of using one epsilon.
  • Use union bounds rather than assuming independence.
  • Keep runtime observations explicit; ASAPPlanner does not import a sketch runtime.
  • Reject point-frequency bounds as proof of TopK membership and reject copying an inner bound onto Hydra.
  • Distinguish mathematical, empirical-input, and future combined parameter configuration; empirical parameter sizing is not wired yet.

Design document: End-to-end accuracy guarantees

Developer guide: Accuracy guarantees developer guide

Verification

  • cargo fmt --all
  • cargo test --workspace
  • cargo clippy --workspace --all-targets -- -D warnings
  • git diff --check

All pass locally. The mapping crate contains 215 passing unit tests, including candidate-construction regressions for accepted TopK and Hydra evidence, plus SQL and PromQL end-to-end coverage.

Independent-review note: this final review was performed by the implementing agent, not an independent reviewer.

Limitations and follow-up

  • TopK remains fail-closed until widened boundary intervals are supplied.
  • Accuracy-targeted Hydra remains fail-closed until shared-grid statistics are supplied.
  • HLL has no confidence-level budget and cannot satisfy EpsilonDelta; adding one requires a new, explicit estimator/parameter contract.
  • Empirical-input parameter sizing and combined mathematical/empirical sizing are not wired yet.
  • The estimator used by a serving implementation must satisfy the contract identity recorded by the selected planner model.
  • Advanced nonlinear/correlation-aware propagation remains follow-up work.

@zzylol
zzylol merged commit 2b330ca into main Aug 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant