Skip to content

feat(post-asap): compose exact operators with summary plans across explicit update/readout boundaries (#171) - #300

Open
zzylol wants to merge 1 commit into
mainfrom
feat/exact-summary-composition-171
Open

feat(post-asap): compose exact operators with summary plans across explicit update/readout boundaries (#171)#300
zzylol wants to merge 1 commit into
mainfrom
feat/exact-summary-composition-171

Conversation

@zzylol

@zzylol zzylol commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

Implements #171 on top of the end-to-end accuracy work merged in #303.

Summary

This PR adds explicit mixed exact/summary execution boundaries to the post-ASAP IR:

  • ExactTransform: an exact operator on the update path whose plain output may feed maintained summary state.
  • ExactPostProcess: an exact operator over a finalized summary readout.
  • Phase validation with typed errors for illegal edges such as readout-under-maintenance.
  • Exact-composition candidates that reference the child's memo group; whole-plan selection commits and costs the parent/child pair.
  • Materialization that preserves shared child identity and validates the completed post-ASAP DAG.
  • Runtime capability and cost hooks; missing capability or statistics retains the conservative pre-ASAP plan.
  • DAG export and explanations for phase assignments and composition decisions.

Integration with #303 accuracy guarantees

The rebased implementation treats phase legality and accuracy legality as independent requirements, in this order:

  1. validate the execution phase and schema;
  2. derive/propagate the result guarantee;
  3. check any requested accuracy target;
  4. admit only legal candidates;
  5. rank legal candidates by recurrence-aware cost.

An exact operator no longer erases an approximate child's guarantee:

  • exact sum uses the existing ExactSum propagation rule;
  • exact min, max, and the supported exact average fold retain the modeled input error magnitude and conservatively compose failure probability through the existing extremum-style union-bound rule;
  • exact inputs remain exact;
  • an approximate input with no registered propagation rule produces a typed AccuracyError instead of being labeled exact;
  • explicit root accuracy filtering fails closed for an unresolved composition candidate.

SummaryNode.guarantee is preserved through relinking/materialization and exported alongside the execution stage. The global selector also retains #303 accuracy-reconciliation use propagation while adding exact-composition child commitments, and retains recurrence-aware CSE decisions.

Phase model

  • SummaryAgg.child accepts UpdateValue or supported exact-accumulator state, never ReadoutValue.
  • SummaryEstimate: SummaryState -> ReadoutValue.
  • ExactTransform: UpdateValue -> UpdateValue.
  • ExactPostProcess: ReadoutValue -> ReadoutValue.
  • An ExactTransform cannot be a final root.
  • KeepPreAsap receives its phase from the validated incoming edge; incompatible shared uses are rejected as ambiguous.

When ordinary summary construction encounters a phase conflict, the strategy reports one conservative KeepPreAsap candidate. Accuracy-rejected candidates remain recorded as rejections and are not confused with phase fallback.

Selection and cost

A composition stores the child target rather than selecting a child locally. PlanSpace::global_selection evaluates compatible child candidates, commits the chosen pair, accounts for shared state, and compares a known cost-units-per-second rate with raw recomputation. Unknown cost inputs do not become zero and cannot make a composition win.

GlobalSelection::materialize then links the selected groups into one phase-validated DAG and memoizes each target, so multiple parents share the same materialized child.

Verification

  • cargo check --workspace — passed after rebase.
  • cargo test -p asap-integration-tests --test exact_composition — 11 passed.
  • Regression coverage verifies phase rejection/fallback, capabilities, missing statistics, shared materialization, DAG export, PromQL composition, and propagation of a child's modeled error magnitude through exact post-processing.
  • A full cargo test --workspace attempt was interrupted by No space left on device while linking the repository's broad test/binary matrix; no test assertion or source compilation failure preceded the environment error.

Deferred

@zzylol

zzylol commented Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Current-main integration review: this PR is not ready to merge as-is after #295 and #299. The conflict is architectural rather than mechanical. #295 now owns the canonical recurrence vocabulary (recurrence::{CostRate, EvaluationRate, UpdateRate, Horizon}) and recurrence-aware selection APIs, while this branch independently introduces duplicate CostRate/EvaluationRate types and a second recurring-cost path in cost_model.rs. #299 now also owns nested-summary legality through ResultGuarantee/AccuracyModel; phase validation must compose with that model rather than independently rejecting or admitting the same nested candidates. Please rebase on current main, reuse the recurrence types and profiles instead of defining parallel units, and make exact composition preserve/propagate the guarantees introduced by #299. The merged implementation should retain the conservative missing-statistics fallback and phase tests from this PR.

…ase boundaries (#171)

Add phase-explicit post-ASAP nodes SummaryExpr::{ExactTransform, ExactPostProcess}
carrying a non-exhaustive ExactOperator::Aggregate payload (never an intact
QueryExpr subtree), plus an ExecutionAvailability {UpdateValue, SummaryState,
ReadoutValue} derivation/validation (post_asap::phase) returning typed
PhaseErrors at construction. construct_summary_agg now validates its edge, so
a maintained summary over a query-time readout falls back conservatively
instead of producing an unexecutable plan.

Add ExactCompositionStrategy (registered in default_strategies) proposing
Replacement::ExactComposition candidates that reference the child target
rather than selecting a child; PlanSpace::global_selection commits the
compatible parent/child pair using the issue's cost-units-per-second
formulas (postprocess/pretransform vs raw-recompute baseline), counts shared
child state once, and GlobalSelection::materialize links the committed
decisions into one validated DAG with shared Rc identity.

Cost hooks: CostModel::mixed_execution_capabilities and
exact_composition_cost_inputs (unknowns stay None, never zero; missing
statistics keep KeepPreAsap). DAG export gains explicit per-node stage,
decision provenance, cost unit and child-decision links (additive).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the feat/exact-summary-composition-171 branch from aafad16 to 07ad2fa Compare August 28, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant