Skip to content

feat(loadtest): --sync-txs for eth_sendRawTransactionSync, plus two runner fixes - #1004

Open
minhd-vu wants to merge 7 commits into
mainfrom
fix/loadtest-sync-send-and-runner-fixes
Open

feat(loadtest): --sync-txs for eth_sendRawTransactionSync, plus two runner fixes#1004
minhd-vu wants to merge 7 commits into
mainfrom
fix/loadtest-sync-send-and-runner-fixes

Conversation

@minhd-vu

@minhd-vu minhd-vu commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Two independent changes, one commit each — the bug fixes first, so they can be split out or cherry-picked if you'd rather they land on their own.

1. fix(loadtest): two bugs on the plain send path

Both pre-date this branch and reproduce without --sync-txs. I hit them while testing the feature below.

Nil gas prices panicked the run. getSuggestedGasPrices returns the cached price and tip cap from six early returns, three of which are RPC-failure paths that run before the cache is first written. On a first-call failure they returned (nil, nil), and configureTransactOpts dereferences the result (tops.GasTipCap.Cmp(...)), so one failed eth_feeHistory or eth_maxPriorityFeePerGas killed the run with a stack trace. suggestMaxFeePerGas likewise returns nil when HeaderByNumber or FeeHistory fails, and that nil was being cached and handed out.

Those returns now go through a cachedPrices helper that substitutes zero for an unset value, and a nil from suggestMaxFeePerGas falls back instead of being cached. The RPC error is already logged at each site so the cause stays visible, and a zero-priced transaction is rejected by the node and recorded as a send error — the same outcome account.go already settles for, and better than losing the run. The helper returns copies: these values outlive the lock and every sending goroutine holds one, so handing out the cached pointers would make any in-place arithmetic by a future caller a silent race on shared state.

--rate-limit -1 hung the run permanently. waitForFinalBlock built its limiter straight from --rate-limit, without the non-positive check the main loop applies ~1000 lines earlier. A negative rate.Limit has tokens that never replenish: the first nonce check took the initial token and every later one blocked on Wait forever, so wg.Wait never returned. Any run with rate limiting disabled and more than one account hung after sending — not a slow wait, an unbounded one. The retry loop also used a bare time.Sleep(5s) × 30 retries, so Ctrl+C was ignored for up to 150 seconds.

Now: no limiter when the rate limit is non-positive, the rl != nil guard already used at the other wait site, time.NewTimer + select on ctx.Done(), and a cancellation check per retry.

Verified against a fake node: the feeHistory-less case exits 0 with Unable to get fee history logged instead of panicking, and the --rate-limit -1 case exits 0.3s after SIGINT with debug logs confirming it was inside the retry loop.

2. feat(loadtest): --sync-txs for eth_sendRawTransactionSync

Sends via EIP-7966 eth_sendRawTransactionSync, which returns when the node has a receipt rather than when the transaction is accepted — so on a preconfirming chain it measures preconfirmation latency directly. Implemented as a send-path option alongside --private-txs rather than a mode, so it composes with every workload that broadcasts raw: transaction, blob, contract-call, recall.

Where bor and the EIP disagree, this follows bor (internal/ethapi/api.go):

EIP-7966 bor this PR
timeout encoding integer *hexutil.Uint64 → hex string only hex by default, --sync-tx-timeout-int for the integer form
error codes 4, 5, 6 only 4; refusals return -38010/-38011/-38013/-38014/-38026 both sets classified
speculative marker none defined "preconfirmation": true, null blockHash marker is authoritative; block-fields fallback otherwise
default wait 2s recommended 20s, clamps above --rpc.txsync.maxtimeout documented, parameter omitted when unset

The encoding one matters most: my first cut sent an integer per the spec text and would have failed against every bor node. Proven both ways against a bor-accurate fake — hex gives 30 receipts, integer gives 8/8 cannot unmarshal non-string into Go value of type hexutil.Uint64.

A SyncTracker on mode.Dependencies logs a summary at run end: receipts split speculative/canonical, reverted, no_status, timeouts, queued, nonce_gaps, rejected, and call latency percentiles.

This changes what the latency numbers mean — recorded request duration becomes time-to-receipt rather than time-to-accept, so --concurrency is the number of transactions in flight. Called out in the usage docs.

Testing

24 new tests across loadtest/mode, loadtest/config and loadtest; -race clean, full suite green, go vet and golangci-lint clean, make gen-doc run.

End-to-end against a fake bor implementing the sync method: submitted:50 receipts:30 speculative:20 canonical:10 timeouts:10 nonce_gaps:10 other_errors:0.

Not tested against a real node. Everything here was verified against fakes built from bor's source and its own test assertions — no run against a live bor with the preconfirmation pipeline enabled, which is the configuration the speculative path actually needs (that pipeline is still unmerged, bor#2373). Against stock bor speculative will read zero, because it requires both a block number and hash before answering.

🤖 Generated with Claude Code

minhd-vu and others added 7 commits September 2, 2026 13:05
…nal wait

Two bugs on the plain send path, both found while testing something else.

getSuggestedGasPrices returns the cached gas price and tip cap from six early
returns, three of which are RPC-failure paths that run before the cache has
ever been written. On a first-call failure they returned (nil, nil), and
configureTransactOpts dereferences the result -- tops.GasTipCap.Cmp(...) -- so
a single failed eth_feeHistory or eth_maxPriorityFeePerGas killed the run with
a stack trace. suggestMaxFeePerGas also returns nil when HeaderByNumber or
FeeHistory fails, and that nil was cached and handed out.

Route those returns through a cachedPrices helper that substitutes zero for an
unset value, and fall back rather than caching a nil from suggestMaxFeePerGas.
The RPC error is already logged at each site, so the cause stays visible, and a
zero-priced transaction is rejected by the node and recorded as a send error --
the same outcome the account pool settles for, and better than losing the run.
The helper returns copies: these values outlive the lock and every sending
goroutine holds one, so handing out the cached pointers would make any in-place
arithmetic by a caller a silent race on shared state.

waitForFinalBlock built its rate limiter straight from --rate-limit without the
non-positive check the main loop applies, so --rate-limit -1 -- the documented
way to remove the limit -- produced a negative rate.Limit whose tokens never
replenish. The first nonce check took the initial token and every later one
blocked on Wait forever, so wg.Wait never returned: any run with rate limiting
disabled and more than one account hung permanently after sending. The retry
loop also slept with a bare time.Sleep, ignoring Ctrl+C for up to 150 seconds.

Skip the limiter when the rate limit is non-positive, guard the wait with the
rl != nil check already used elsewhere, and replace the sleep with a timer that
selects on ctx.Done(), plus a cancellation check per retry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Send transactions with eth_sendRawTransactionSync (EIP-7966) instead of
eth_sendRawTransaction. That call returns when the node has a receipt rather
than when the transaction is accepted, so on a chain that preconfirms it
measures preconfirmation latency directly.

Implemented as a send-path option rather than a mode, alongside --private-txs,
so it composes with every workload that broadcasts raw: transaction, blob,
contract-call and recall.

Aligned with bor rather than the EIP text where the two disagree:

  - bor takes the timeout as *hexutil.Uint64, which unmarshals only from a
    quoted hex quantity and rejects a bare JSON number. The value therefore
    goes out as hex by default; --sync-tx-timeout-int selects the integer form
    the EIP describes, which spec-literal servers require instead.
  - bor implements only error code 4 (timeout) of the EIP's 4/5/6. Anything it
    refuses fails in SubmitTransaction before the wait and returns bor's own
    codes, so -38011 is counted as a nonce gap and -38010, -38013, -38014 and
    -38026 as rejections rather than unknown errors.
  - bor marks a preconfirmed receipt with "preconfirmation": true and a null
    blockHash, and that marker is authoritative when present. Other nodes get
    the block-fields reading, since EIP-7966 defines no such field.
  - bor defaults the wait to 20s, not the EIP's recommended 2s, and clamps
    anything above --rpc.txsync.maxtimeout instead of rejecting it.

A SyncTracker on mode.Dependencies aggregates outcomes and logs a summary at
the end of the run: receipts split speculative/canonical, reverted, no_status,
timeouts, queued, nonce_gaps, rejected, and call latency percentiles. Pair with
--wait-for-receipt to confirm canonical inclusion after the synchronous call
returns, which measures the speculative and canonical receipts separately.

Note this changes what the latency numbers mean: the recorded request duration
becomes time to receipt rather than time to accept, so --concurrency is the
number of transactions in flight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Review follow-up on the two commits before this one.

The send chain was duplicated across four modes, and --sync-txs added a
fourth identical branch to each copy. Extract SendSignedTransaction next to
SendRawTransactionPrivate, preserving the original precedence exactly
(output-raw, sync, private, plain), so the next send method is wired in once
rather than four times. Call-only handling stays per mode, where it genuinely
differs.

Guard the block number read in SyncTracker.Record. Because the
"preconfirmation" marker is authoritative over the block-fields heuristic, a
receipt with "preconfirmation": false and no blockNumber is classified
canonical, and (*hexutil.Big)(nil).ToInt().Uint64() panics -- killing the run
from inside every sending goroutine. TestSyncReceiptClassification already
described that receipt shape but never drove it through Record;
TestSyncTrackerCanonicalReceiptWithoutBlockNumber now does, and fails with a
nil dereference without the guard.

Also: drop the redundant SyncTracker nil check in postLoadTest, since Stats is
a no-op on a nil tracker, and retitle the comment that had been left labelling
the new block as preconf output; drop the unreachable timer.Stop after the
timer has fired; preallocate the durations slice in Stats.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Decode the eth_sendRawTransactionSync response into raw JSON before
unmarshalling the SyncReceipt, and at verbosity 700 log each raw receipt
verbatim (or the RPC error with its code and data on failure) so the
node's answers can be checked against eth_getTransactionReceipt later.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t-for-receipt

Poll eth_getTransactionReceipt into raw JSON via the new
util.WaitReceiptRaw so --wait-for-receipt trace-logs each receipt
verbatim at verbosity 700, sharing the log shape with --sync-txs
through the now-exported mode.LogReceiptTrace. The wait still blocks
the sending worker goroutine.

Add --receipt-poll-interval to poll at a fixed interval instead of
exponential backoff; in that mode polling is bounded only by the
receipt timeout and --receipt-retry-max is ignored. Also correct the
sync-txs docs to reference --pretty-logs=false, which is the actual
flag for JSON log output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mirror the loadtest gas flags in the fund command: on EIP-1559 chains
--gas-price forces the max fee per gas and --priority-gas-price the gas
tip cap; on legacy chains --gas-price forces the gas price and the tip
is ignored with a warning. The overrides apply to the multicall3,
funder-contract (including the prefund tx via util.SendTx), and ERC20
mint/approve paths. Also give each multicall3 batch goroutine its own
TransactOpts copy since the helpers mutate tops.Value concurrently.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The follow loop only advanced its cursor when the whole range printed
successfully, so a mid-range failure re-fetched and re-printed blocks
that had already been output. This happened routinely behind a
round-robin load balancer: eth_blockNumber could hit a node ahead of
the one serving eth_getBlockByNumber, which returned null for the
newest blocks and made the print loop error partway through.

writeBlockRange now treats null responses as blocks not yet available
on the serving node, stops printing before the first missing block so
it can be retried, and reports the last printed block so the caller
advances its cursor past printed blocks even on error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants