feat(loadtest): --sync-txs for eth_sendRawTransactionSync, plus two runner fixes - #1004
Open
minhd-vu wants to merge 7 commits into
Open
feat(loadtest): --sync-txs for eth_sendRawTransactionSync, plus two runner fixes#1004minhd-vu wants to merge 7 commits into
minhd-vu wants to merge 7 commits into
Conversation
…nal wait Two bugs on the plain send path, both found while testing something else. getSuggestedGasPrices returns the cached gas price and tip cap from six early returns, three of which are RPC-failure paths that run before the cache has ever been written. On a first-call failure they returned (nil, nil), and configureTransactOpts dereferences the result -- tops.GasTipCap.Cmp(...) -- so a single failed eth_feeHistory or eth_maxPriorityFeePerGas killed the run with a stack trace. suggestMaxFeePerGas also returns nil when HeaderByNumber or FeeHistory fails, and that nil was cached and handed out. Route those returns through a cachedPrices helper that substitutes zero for an unset value, and fall back rather than caching a nil from suggestMaxFeePerGas. The RPC error is already logged at each site, so the cause stays visible, and a zero-priced transaction is rejected by the node and recorded as a send error -- the same outcome the account pool settles for, and better than losing the run. The helper returns copies: these values outlive the lock and every sending goroutine holds one, so handing out the cached pointers would make any in-place arithmetic by a caller a silent race on shared state. waitForFinalBlock built its rate limiter straight from --rate-limit without the non-positive check the main loop applies, so --rate-limit -1 -- the documented way to remove the limit -- produced a negative rate.Limit whose tokens never replenish. The first nonce check took the initial token and every later one blocked on Wait forever, so wg.Wait never returned: any run with rate limiting disabled and more than one account hung permanently after sending. The retry loop also slept with a bare time.Sleep, ignoring Ctrl+C for up to 150 seconds. Skip the limiter when the rate limit is non-positive, guard the wait with the rl != nil check already used elsewhere, and replace the sleep with a timer that selects on ctx.Done(), plus a cancellation check per retry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Send transactions with eth_sendRawTransactionSync (EIP-7966) instead of
eth_sendRawTransaction. That call returns when the node has a receipt rather
than when the transaction is accepted, so on a chain that preconfirms it
measures preconfirmation latency directly.
Implemented as a send-path option rather than a mode, alongside --private-txs,
so it composes with every workload that broadcasts raw: transaction, blob,
contract-call and recall.
Aligned with bor rather than the EIP text where the two disagree:
- bor takes the timeout as *hexutil.Uint64, which unmarshals only from a
quoted hex quantity and rejects a bare JSON number. The value therefore
goes out as hex by default; --sync-tx-timeout-int selects the integer form
the EIP describes, which spec-literal servers require instead.
- bor implements only error code 4 (timeout) of the EIP's 4/5/6. Anything it
refuses fails in SubmitTransaction before the wait and returns bor's own
codes, so -38011 is counted as a nonce gap and -38010, -38013, -38014 and
-38026 as rejections rather than unknown errors.
- bor marks a preconfirmed receipt with "preconfirmation": true and a null
blockHash, and that marker is authoritative when present. Other nodes get
the block-fields reading, since EIP-7966 defines no such field.
- bor defaults the wait to 20s, not the EIP's recommended 2s, and clamps
anything above --rpc.txsync.maxtimeout instead of rejecting it.
A SyncTracker on mode.Dependencies aggregates outcomes and logs a summary at
the end of the run: receipts split speculative/canonical, reverted, no_status,
timeouts, queued, nonce_gaps, rejected, and call latency percentiles. Pair with
--wait-for-receipt to confirm canonical inclusion after the synchronous call
returns, which measures the speculative and canonical receipts separately.
Note this changes what the latency numbers mean: the recorded request duration
becomes time to receipt rather than time to accept, so --concurrency is the
number of transactions in flight.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review follow-up on the two commits before this one. The send chain was duplicated across four modes, and --sync-txs added a fourth identical branch to each copy. Extract SendSignedTransaction next to SendRawTransactionPrivate, preserving the original precedence exactly (output-raw, sync, private, plain), so the next send method is wired in once rather than four times. Call-only handling stays per mode, where it genuinely differs. Guard the block number read in SyncTracker.Record. Because the "preconfirmation" marker is authoritative over the block-fields heuristic, a receipt with "preconfirmation": false and no blockNumber is classified canonical, and (*hexutil.Big)(nil).ToInt().Uint64() panics -- killing the run from inside every sending goroutine. TestSyncReceiptClassification already described that receipt shape but never drove it through Record; TestSyncTrackerCanonicalReceiptWithoutBlockNumber now does, and fails with a nil dereference without the guard. Also: drop the redundant SyncTracker nil check in postLoadTest, since Stats is a no-op on a nil tracker, and retitle the comment that had been left labelling the new block as preconf output; drop the unreachable timer.Stop after the timer has fired; preallocate the durations slice in Stats. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Decode the eth_sendRawTransactionSync response into raw JSON before unmarshalling the SyncReceipt, and at verbosity 700 log each raw receipt verbatim (or the RPC error with its code and data on failure) so the node's answers can be checked against eth_getTransactionReceipt later. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t-for-receipt Poll eth_getTransactionReceipt into raw JSON via the new util.WaitReceiptRaw so --wait-for-receipt trace-logs each receipt verbatim at verbosity 700, sharing the log shape with --sync-txs through the now-exported mode.LogReceiptTrace. The wait still blocks the sending worker goroutine. Add --receipt-poll-interval to poll at a fixed interval instead of exponential backoff; in that mode polling is bounded only by the receipt timeout and --receipt-retry-max is ignored. Also correct the sync-txs docs to reference --pretty-logs=false, which is the actual flag for JSON log output. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mirror the loadtest gas flags in the fund command: on EIP-1559 chains --gas-price forces the max fee per gas and --priority-gas-price the gas tip cap; on legacy chains --gas-price forces the gas price and the tip is ignored with a warning. The overrides apply to the multicall3, funder-contract (including the prefund tx via util.SendTx), and ERC20 mint/approve paths. Also give each multicall3 batch goroutine its own TransactOpts copy since the helpers mutate tops.Value concurrently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The follow loop only advanced its cursor when the whole range printed successfully, so a mid-range failure re-fetched and re-printed blocks that had already been output. This happened routinely behind a round-robin load balancer: eth_blockNumber could hit a node ahead of the one serving eth_getBlockByNumber, which returned null for the newest blocks and made the print loop error partway through. writeBlockRange now treats null responses as blocks not yet available on the serving node, stops printing before the first missing block so it can be retried, and reports the last printed block so the caller advances its cursor past printed blocks even on error. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent changes, one commit each — the bug fixes first, so they can be split out or cherry-picked if you'd rather they land on their own.
1.
fix(loadtest): two bugs on the plain send pathBoth pre-date this branch and reproduce without
--sync-txs. I hit them while testing the feature below.Nil gas prices panicked the run.
getSuggestedGasPricesreturns the cached price and tip cap from six early returns, three of which are RPC-failure paths that run before the cache is first written. On a first-call failure they returned(nil, nil), andconfigureTransactOptsdereferences the result (tops.GasTipCap.Cmp(...)), so one failedeth_feeHistoryoreth_maxPriorityFeePerGaskilled the run with a stack trace.suggestMaxFeePerGaslikewise returns nil whenHeaderByNumberorFeeHistoryfails, and that nil was being cached and handed out.Those returns now go through a
cachedPriceshelper that substitutes zero for an unset value, and a nil fromsuggestMaxFeePerGasfalls back instead of being cached. The RPC error is already logged at each site so the cause stays visible, and a zero-priced transaction is rejected by the node and recorded as a send error — the same outcomeaccount.goalready settles for, and better than losing the run. The helper returns copies: these values outlive the lock and every sending goroutine holds one, so handing out the cached pointers would make any in-place arithmetic by a future caller a silent race on shared state.--rate-limit -1hung the run permanently.waitForFinalBlockbuilt its limiter straight from--rate-limit, without the non-positive check the main loop applies ~1000 lines earlier. A negativerate.Limithas tokens that never replenish: the first nonce check took the initial token and every later one blocked onWaitforever, sowg.Waitnever returned. Any run with rate limiting disabled and more than one account hung after sending — not a slow wait, an unbounded one. The retry loop also used a baretime.Sleep(5s)× 30 retries, so Ctrl+C was ignored for up to 150 seconds.Now: no limiter when the rate limit is non-positive, the
rl != nilguard already used at the other wait site,time.NewTimer+selectonctx.Done(), and a cancellation check per retry.Verified against a fake node: the feeHistory-less case exits 0 with
Unable to get fee historylogged instead of panicking, and the--rate-limit -1case exits 0.3s after SIGINT with debug logs confirming it was inside the retry loop.2.
feat(loadtest):--sync-txsforeth_sendRawTransactionSyncSends via EIP-7966
eth_sendRawTransactionSync, which returns when the node has a receipt rather than when the transaction is accepted — so on a preconfirming chain it measures preconfirmation latency directly. Implemented as a send-path option alongside--private-txsrather than a mode, so it composes with every workload that broadcasts raw:transaction,blob,contract-call,recall.Where bor and the EIP disagree, this follows bor (
internal/ethapi/api.go):*hexutil.Uint64→ hex string only--sync-tx-timeout-intfor the integer form-38010/-38011/-38013/-38014/-38026"preconfirmation": true, nullblockHash--rpc.txsync.maxtimeoutThe encoding one matters most: my first cut sent an integer per the spec text and would have failed against every bor node. Proven both ways against a bor-accurate fake — hex gives 30 receipts, integer gives 8/8
cannot unmarshal non-string into Go value of type hexutil.Uint64.A
SyncTrackeronmode.Dependencieslogs a summary at run end: receipts split speculative/canonical, reverted, no_status, timeouts, queued, nonce_gaps, rejected, and call latency percentiles.This changes what the latency numbers mean — recorded request duration becomes time-to-receipt rather than time-to-accept, so
--concurrencyis the number of transactions in flight. Called out in the usage docs.Testing
24 new tests across
loadtest/mode,loadtest/configandloadtest;-raceclean, full suite green,go vetandgolangci-lintclean,make gen-docrun.End-to-end against a fake bor implementing the sync method:
submitted:50 receipts:30 speculative:20 canonical:10 timeouts:10 nonce_gaps:10 other_errors:0.Not tested against a real node. Everything here was verified against fakes built from bor's source and its own test assertions — no run against a live bor with the preconfirmation pipeline enabled, which is the configuration the speculative path actually needs (that pipeline is still unmerged, bor#2373). Against stock bor
speculativewill read zero, because it requires both a block number and hash before answering.🤖 Generated with Claude Code