Skip to content

feat(payload): add shared resource metering evaluator - #4673

Open
niran wants to merge 3 commits into
mainfrom
niran/resource-metering-evaluator
Open

feat(payload): add shared resource metering evaluator#4673
niran wants to merge 3 commits into
mainfrom
niran/resource-metering-evaluator

Conversation

@niran

@niran niran commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a shared, versioned resource-metering schedule and decision model used by both payload builders.
  • Per-dimension dryRun (omitted = enforce), fail-open arithmetic, and transaction-versus-block throttle scope.
  • Move metering provider and rejection-cache ownership into the payload crate, keeping builder-core re-exports.

Stacked replacement for the shared evaluator slice of #4428. Later PRs in this stack add observations, native admission, Flashblocks admission, and wall-clock retirement.

Test plan

  • cargo test --lib -p base-execution-payload-builder
  • Confirm later stack PRs still compile against this evaluator API

type=routine
risk=low
impact=sev5

Define one versioned schedule and decision model for both payload builders, including per-dimension dry-run, fail-open arithmetic, and transaction-versus-block scope. Move provider and rejection-cache ownership into the payload crate while preserving builder-core re-exports.
@cb-heimdall

Copy link
Copy Markdown
Collaborator

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

/// Independently budgeted resource dimensions.
pub dimensions: Vec<ResourceMeteringDimension>,
#[serde(skip)]
operation_index: HashMap<String, Vec<(usize, u64, u64)>>,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: operation_index uses HashMap (non-deterministic iteration order). This is fine here since evaluate() only does key lookups (not iteration), but worth noting that the Serialize derive on ResourceMeteringSchedule will skip this field (#[serde(skip)]), so serialization determinism is preserved. If a future change iterates operation_index in a consensus-sensitive path, it should switch to BTreeMap.

Comment on lines +149 to +153
);
if decision.should_exclude() {
self.record_decision(tx_hash, &decision);
}
(simulated, decision)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: predict only records decisions where should_exclude() is true (enforced throttles). Dry-run throttle observations at the prediction stage are silently dropped — the metric/log won't fire until decide_executed runs after execution. This means if a caller uses predict to skip execution entirely (which it can't today since should_exclude() is false for dry-run), the observation would be lost.

This is fine given the current flow where CalculationFailed and dry-run Throttle both proceed to execution, but it's worth documenting explicitly that predict intentionally under-reports dry-run observations, since a future caller might expect the metrics to be recorded here regardless of the exclude decision.

@github-actions

Copy link
Copy Markdown
Contributor

Review Summary

PR: feat(payload): add shared resource metering evaluator

This PR introduces a versioned resource-metering schedule and decision model (ResourceMeteringSchedule, ResourceMeteringConfig) into the base-execution-payload-builder crate, moving the MeteringProvider trait, RejectionCache, and related types from builder-core to the payload crate. The change is well-structured with thorough test coverage.

Block-Production Safety Assessment

This PR is block-production-sensitive (touches metering and payload assembly paths). After reviewing against the block production review guide:

  • Fail-open design is consistently applied: CalculationFailed never excludes transactions, arithmetic overflow in add_to / apply_accounted_usage is caught and fails open, and missing meter data produces zero usage. This means a misconfigured schedule cannot halt payload construction.
  • Dry-run dimension support: Dimensions can be observed without excluding, providing a safe rollout mechanism.
  • No new panics in hot paths: All arithmetic uses checked_* operations. No unwrap()/expect() in non-test code paths.
  • Schedule loading is startup-only: from_file reads from disk only in from_parts, not during block building.
  • No new I/O boundaries: The evaluator operates in-process on data already available to the builder.

No critical block-production findings.

Minor Findings

  1. HashMap in operation_index (resource_metering.rs:45): Safe for current use (key lookups only, #[serde(skip)]), but worth noting for future maintainers if iteration is ever added in a deterministic context.

  2. predict observation gap (config.rs:149-153): Dry-run throttle observations at prediction time are intentionally not recorded — only enforced excludes fire the metric/log. The deferred recording in decide_executed covers this, but the asymmetry could surprise future callers.

What Looks Good

  • Comprehensive validation at schedule compile time (name constraints, duplicate detection, noop checks, limit ordering)
  • Atomic add_to semantics preventing partial cumulative writes
  • Clean separation between file-format DTOs and runtime types
  • Thorough test coverage including overflow, fail-open, dry-run, and edge cases
  • Bounded metric cardinality (dimension names from operator-controlled schedule, max 128)

@github-actions

github-actions Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Base Std historical fork tests

Fork Result Passed Failed Skipped base/base base-anvil base-std
Beryl pass 616 0 13 6a30feab cecddfa5 4658f1b7
Cobalt pass 721 0 14 6a30feab fb00db40 e30b3421

View run

@github-actions

Copy link
Copy Markdown
Contributor

Caution

This PR may regress performance. 1 benchmark(s) slower by more than 10% beyond the noise band: execution/Open 1024 nodes - 4096 nodes (+49.1%).

Benchmark results (advisory)

Median time on the PR head versus the base branch, measured on the same host. Wall-clock, so a change is only flagged when it clears ±10% and the confidence intervals do not overlap. Only benchmarks past the ±10% threshold (plus new or dropped ones) are listed. This check never blocks a merge.

Benchmark Base Head Δ median
execution/Delete 16 nodes - 65,536 nodes 5.38 ms 4.47 ms -16.9% ✅ faster
execution/Insertion - 65,536 nodes 34.88 ms 31.33 ms -10.2% ✅ faster
execution/Open 1024 nodes - 4096 nodes 37.80 µs 56.34 µs +49.1% ⚠️ slower

46 benchmark(s) within ±10% omitted.

View run · Re-run benchmarks

@niran
niran requested a review from 0x00101010 August 25, 2026 22:43
Comment thread crates/execution/payload/src/config.rs Outdated
}

/// Executed admission check. Records the final decision.
pub fn decide_executed(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit confused by the naming here, who calls it and what does it do exactly?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to simulated_admission / executed_admission so they pair with simulated_sample and name the data source instead of a vague verb.

Payload builders call simulated_admission before EVM execution and skip when should_exclude() is true. The returned sample is passed to executed_admission after execution. Sequencer txs still go through unthrottled_usage / account_unthrottled.

Comment thread crates/execution/payload/src/config.rs
@@ -0,0 +1,1565 @@
//! Versioned resource-metering schedules and their transaction-cost evaluator.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file purely new code or is it copies / moved from somewhere existing?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly new. The evaluator, schedule JSON, samples, and decision types in this file did not exist before. What moved from builder-core in this PR is MeteringProvider (metering.rs) and RejectionCache (rejection_cache.rs). The old wall-clock execution-time throttle stays in builder-core until #4677; this file is not a copy of that.

…ed admission

The old names hid that these are meterBundle vs post-state admission checks, not a forecast or a generic decision helper.
@github-actions

github-actions Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

📊 14 benchmark(s) added — baselines recorded (deterministic instruction counts under Valgrind). Per-change deltas will appear on future PRs, once these land on the base branch. View run

Benchmark details (14)
Benchmark Base (target) Head (this PR) Δ instructions
batch_queue/drain/drain_cached_span_batches 242,027 🆕 new
batch_transaction/encode_in_place/encode_in_place 4,199,759 🆕 new
batch_transaction/temporary_frame_buffers/temporary_frame_buffers 8,408,350 🆕 new
flashblock_decode/decode/brotli 3,296,484 🆕 new
flashblock_decode/decode/plain_json 2,280,194 🆕 new
flz/compress_len/real_contract_call 43,148 🆕 new
flz/compress_len/synthetic_0 38,205 🆕 new
flz/compress_len/synthetic_1 54,682 🆕 new
flz/compress_len/synthetic_2 147,976 🆕 new
flz/data_gas 43,059 🆕 new
flz/tx_estimated_size 43,056 🆕 new
frame_parse/decode/single_4kib 1,031 🆕 new
frame_parse/parse_frames/few_large 1,053,062 🆕 new
frame_parse/parse_frames/many_small 154,763 🆕 new

.copied()
.unwrap_or_default()
.checked_add(transaction_cost)
.ok_or(ResourceThrottlingCheckError::ArithmeticOverflow)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The early ? return from checked_add overflow exits the loop before later dimensions are examined. If dimension N has a cumulative overflow but dimension N+1 has an enforced transaction-scope violation, the method returns ArithmeticOverflow (→ CalculationFailed → fail-open) instead of the permanent transaction-scope rejection.

This breaks the priority invariant documented in the method doc comment ("enforced transaction-scope, then enforced block-scope, then the first dry-run overrun"). In practice u128 cumulative overflow is near-impossible, but the early return silently downgrades a permanent eviction to a fail-open pass.

Consider accumulating the overflow like the other error classes instead of returning early:

Suggested change
.ok_or(ResourceThrottlingCheckError::ArithmeticOverflow)?;
.ok_or(ResourceThrottlingCheckError::ArithmeticOverflow);
let used = match used {
Ok(used) => used,
Err(_) => continue,
};

This way the loop continues and can still find an enforced transaction-scope violation in a later dimension.

@github-actions

Copy link
Copy Markdown
Contributor

Review Summary

Well-structured PR that introduces a shared resource-metering evaluator with versioned schedules and moves metering/rejection-cache ownership to the payload crate. The fail-open design throughout is the right approach for block-production safety.

Findings

Minor (1 finding):

  • check() early return from arithmetic overflow can mask transaction-scope violations (resource_metering.rs:443): The ? on checked_add overflow exits the loop before later dimensions are examined. If a later dimension has an enforced transaction-scope violation (permanent eviction), it would be missed and the transaction would fail-open instead. Practically unreachable with u128 arithmetic, but the early return breaks the priority invariant documented in the method. See inline comment for a suggested fix.

Block Production Assessment

This PR is block-production-sensitive (metering, tx selection). The fail-open design is consistently applied:

  • is_active() guard on every path makes disabled metering a complete no-op
  • CalculationFailed always fails open (never halts payload construction)
  • add_to overflow fails open
  • Missing metering data fails open with zero usage
  • Sequencer transactions use account_unthrottled which cannot block inclusion

No block-production halt/stall risks identified.

Other Observations (no action needed)

  • Schedule file is read synchronously in from_file — acceptable since this only runs at startup via from_parts.
  • run_pending_tasks was previously #[cfg(any(test, feature = "test-utils"))] in the old rejection cache and is now unconditionally public. The moka docs recommend periodic maintenance calls, so this is reasonable.
  • Test coverage is thorough: overflow fail-open paths, dry-run vs enforce, priority ordering of enforced tx-scope over block-scope, state-effect counting, and executed-vs-simulated sample overlay are all tested.

…d_usage

Name the two limit checks after the sample they use. Neither method accumulates block usage; callers apply that after include.
@github-actions

Copy link
Copy Markdown
Contributor

Review Summary

This PR introduces a shared, versioned resource-metering schedule and evaluator for payload builders, with well-designed fail-open semantics throughout. The code moves metering provider and rejection cache ownership to the payload crate and adds comprehensive validation, evaluation, and throttling logic.

Block Production Safety

The PR is block-production-sensitive (touches metering and payload assembly). After tracing all error paths in ResourceMeteringConfig, the fail-open design is consistent: CalculationFailed never excludes transactions, overflow in add_to/apply_accounted_usage is absorbed, and sequencer transactions always commit via account_unthrottled. No block-production halt/stall risks identified.

Existing Inline Findings (from prior review)

Three inline comments were previously posted and remain relevant:

  1. check() early ? return on ArithmeticOverflow (resource_metering.rs:443) — breaks the documented priority invariant (enforced tx-scope > enforced block-scope > dry-run). An overflow in dimension N short-circuits before a later dimension N+1 can report an enforced transaction-scope violation. The overflow becomes CalculationFailed → fail-open, potentially masking a permanent eviction.

  2. HashMap in operation_index (resource_metering.rs) — fine for lookup-only use; noted that future iteration in consensus-sensitive paths should switch to BTreeMap.

  3. check_simulated_usage under-reports dry-run observations (config.rs) — only records when should_exclude() is true, so dry-run throttles at simulation stage are silently dropped until post-execution recording.

Additional Observations (no new inline comments needed)

  • Uncompiled schedule silently drops operation costs: evaluate() works on the operation_index built by compile(). A schedule created via new() without calling compile() will produce costs from base_gas_weight only, silently ignoring all operation prices. Production paths (from_json, from_file, from_parts) all compile automatically, so this is an internal API footgun rather than a production risk. Consider a debug assertion or a compiled flag if direct construction becomes more common in later stack PRs.

  • Rejection cache run_pending_tasks visibility change: Was #[cfg(any(test, feature = "test-utils"))] in the old location, now unconditionally pub. Harmless but intentional widening of the API surface.

Overall the design is solid with thorough test coverage of edge cases (overflow, fail-open, dry-run vs enforce, priority ranking). The primary actionable item from prior reviews is the check() early return on overflow (finding #1 above).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants