Label everything the stacks can name, and add the axis they could not - #37
Merged
zmaril merged 4 commits intoAug 27, 2026
Merged
Conversation
On bun's release build 0.75% of every sample was unattributed, and nearly all of it was one thing: loop strength reduction and the scalar evolution analysis it drives. LLVM enters that pass through a free function, and the unwinder does not get past it -- so the outermost frame on the stack is `ReduceLoopStrength` and nothing above it says the optimiser is running. The dev build did not show this, because at `-O0` the pass never runs. The release build is where the census's time actually is, and this was the last bucket in it with nothing in it but a shrug. Release goes 0.75% -> 0.06% unattributed, and the samples land in `optimize / loop`, taking it from 7.0% of the build to 8.1%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015iY4ffQbZC8ix4g4fs1BR5
cratebank previewPreview removed because this pull request is closed. |
The third level was thin where it mattered. A release build of bun had a
quarter of its samples sitting at a phase with no detail under it -- and
`optimize`, which is 45% of that build, was the worst of them at 10.7% bare.
Sweeping every phase at once rather than one at a time, what was in there
was not noise. It was passes with names.
* `optimize` gains `sccp`, `globalopt`, `callgraph`, `cse`, `dce` and
`lower`: 10.7% -> 0.7%.
* `machine_code` gains `analysis`, `prepare` and `emit`, and its existing
`regalloc`, `frame` and `machine_opt` gain the passes they had always
meant: 5.0% -> 1.1%.
* `macro_expand` splits into `expand`, `reparse` and `proc_macro` -- the
last being the macro's own code, which is the number to look at before
blaming a build's macros in general.
* `mir_build` splits into `thir`, `build`, `match_check` and `unsafety`;
`metadata_encode` is mostly `mir_transform`, because encoding forces
`optimized_mir` for everything a dependent might inline; `codegen`'s
leftovers were `bitcode` and, once `codegen` was allowed to carry them,
the same `debug_info` and `layout` its neighbours already had.
* `startup` splits into `dyld`, `ctxt`, `thread` and `llvm_init`, and
`resolve`, `analysis`, `lint` and `ast_lowering` get theirs.
A quarter of the build -> 3.6%, and what remains is the right leftover: the
pass manager dispatching between passes, and the allocator. Neither gets a
marker, deliberately -- matching `AnalysisManager` once charged a quarter of
the optimiser to the dispatcher instead of to the pass that asked.
Two of these were worth measuring rather than assuming. `emit` -- the object
file reaching the disk, 2.3% of a debug build in `write` alone -- is matched
innermost and could have taken samples from `asm`; it took none, because
every one of those writes already had no pass above it. And `codegen`
carrying `debug_info` moved 4.3% of a debug build out of the leftover bucket
without touching what `ir_build` and `machine_code` already reported.
The probe grows a `*` mode for the question this pass kept asking: what has
no detail, everywhere, in one walk of the profile instead of one per phase.
`phase.rs` becomes a directory because the detail table is now the biggest
of the three and was pushing the file past its limit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015iY4ffQbZC8ix4g4fs1BR5
The phases nothing had ever been said about, said: `parse` splits into `path` and `tokens`, `trait_solve` into `select` and `normalize`, `const_eval` into `interpret` and the solving it drags in, `resolve` gains `late`, `ir_build` gains `declare` -- the symbol names, ABIs and attributes a codegen unit needs before any of it is filled in. Two cross-cutting names, because two costs are cross-cutting. `dep_graph` is the incremental dependency graph being written *as a phase runs* rather than at the end: a third of everything type checking did that type checking itself could not explain. `hash` is the stable hashing the same system needs. Neither belongs to one phase and both were invisible inside all of them. And `metadata_encode` finally gets `encode`, which is worth having only for the contrast with the `mir_transform` beside it: writing a crate's metadata spends more time running MIR passes than writing anything. 2.8% of a release build now has no detail, from a quarter before this and the commit before it. One name was measured and taken back out. `emit` -- the write syscall -- reads well in `machine_code`, where it took nothing from `asm` because those writes had no pass above them. Inside `link` the same marker took 11,860 samples off `archive` and left it 732: there the writes *are* the archive being built, and `archive` is the better name for them. A leaf marker is only safe where the leaf is all there is, and that is now written next to it. What is left unnamed is the pass manager dispatching between passes, the allocator, and `memcpy`. None of them will get a marker. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015iY4ffQbZC8ix4g4fs1BR5
Segment, phase and detail are three grains of one question -- why was the compiler here -- and they nest. This is a different question that does not nest with them, so it gets a column rather than a fourth level: what was running. It is worth a column because the answer is "not a compiler pass" 24% of the time on a release build and 45% on a debug one, and none of that was visible. It is spread across two dozen phases in slices too thin to notice, and the phase it is spread across is never wrong: `optimize` really was optimising, inside a `DenseMap`. The two builds have completely different shapes, which is the finding: concern release debug pass_manager 6.5% 0.8% alloc 5.0% 3.9% hashmap 4.8% 5.9% memcpy 2.8% 7.0% intern 2.1% 5.7% query 1.3% 4.5% io 0.8% 11.6% hash 0.2% 3.5% total 24.2% 45.0% A release build's largest single concern is LLVM choosing what to run next. A debug build's is the machine waiting on its disk -- 11.6% of every sample, which no compiler flag addresses. # The rule, and why it is the weak one A concern is the leaf frame or nothing. Not the innermost match on the stack; the leaf. The stronger rule -- a named concern claiming everything beneath it, so a `memcpy` inside a hasher counts as hashing -- was measured and rejected. Across ten concerns it changed nothing worth having: every one moved by less than a percentage point except `dep_graph`, which moved seventeen-fold. That one number is the entire case for it, and it is the one to trust least, because claiming a subtree from an enclosing frame is how `pass_manager` first measured 57% of a build. The pass manager is on the stack of every sample the optimiser takes, so "matched anywhere" meant "matched always". Leaf-anchored has no such failure mode and no parameter to tune, and a test pins the case: a sample inside `InstCombine` is `instcombine` with no concern, even with the pass manager one frame up. # What moved, and what did not `dep_graph` and `hash` were details until now, which was the wrong column: they are machinery, not what the phase set out to do, and as details they competed with the phase's real details for one slot. They move here. That is the only thing that changed in the phase tables, and it is accounted for exactly: three `hash` detail rows, 852 samples, redistributed into `lower`, `typeck` and `encode` in the same phases they were already in. 852 out, 852 in, nothing crossed a phase boundary. Nothing else moved a sample, which is the property the axis has to have -- summing over `concern` gives the phases back. And `dep_graph` turns out to be almost nothing: 0.3% of a debug build and none at all of a release one. It is the concern that prompted this and the smallest thing in it. What it cannot answer -- what incrementality costs -- wants a build run twice and the elapsed times compared, which is a thing this census is shaped to do and no attribution of one run can replace. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015iY4ffQbZC8ix4g4fs1BR5
zmaril
marked this pull request as ready for review
August 27, 2026 19:27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A follow-up to #36, measured after it merged. Four commits: leave nothing in
the phase tables that a stack could have named, then add the column for the
thing the phase tables structurally cannot say.
1. The optimiser arriving without its pass manager
#36 drove
unattributedfrom 5.1% to 0.09% — but that was measured on bun'sdev build. On the release build it sat at 0.75%, nearly all of it
loop strength reduction: LLVM enters that pass through a free function and
the unwinder does not get past it, so there is no pass manager on the stack
and no
LLVMRustOptimizeabove it. At-O0the pass never runs, which iswhy the dev build showed none of it. Release: 0.75% → 0.06%.
2. The third level, everywhere it was thin
A quarter of a release build's samples sat at a phase with no detail under
them. Sweeping every phase at once rather than one at a time, what was in
there was passes with names.
optimizemachine_codemetadata_encodemir_buildmacro_expandThree of the new details answer questions nobody could ask before:
macro_expand / proc_macroseparates the macro's own code (syn,quote) from what rustc did around it;metadata_encode / mir_transformshows that writing a crate's metadata spends more time running MIR passes
than writing anything;
ir_build / declareis the symbol names and ABIspaid before any body is filled in.
3. The other axis
Segment, phase and detail are three grains of one question — why was the
compiler here — and they nest. There is a different question that does not
nest with them, so it gets a column rather than a fourth level: what was
running, taken from the leaf frame alone, null when the sample was inside a
compiler pass doing that pass's own work.
It earns the column because the answer is "not a compiler pass" 24% of the
time on a release build and 45% on a debug one, and none of it was visible:
it is spread across two dozen phases in slices too thin to notice, and the
phase it is spread across is never wrong —
optimizereally was optimising,inside a
DenseMap.pass_managerallochashmapmemcpyinternqueryiohashdep_graphA release build's largest single concern is LLVM choosing what to run next. A
debug build's is the machine waiting on its disk, which no compiler flag
addresses.
The rule is the weak one, on purpose
A concern is the leaf frame or nothing. The stronger rule — a named
concern claiming everything beneath it — was measured and rejected: across
ten concerns it moved every one by less than a percentage point except
dep_graph, which moved seventeen-fold. That single number is its entirecase, and it is the one to trust least, because claiming a subtree from an
enclosing frame is how
pass_managerfirst measured 57% of a build. Thepass manager is on the stack of every sample the optimiser takes, so "matched
anywhere" meant "matched always".
A test pins it: a sample inside
InstCombineisinstcombinewith noconcern, even with the pass manager one frame up.
dep_graphis the joke of itThe concern that prompted the axis is the smallest thing in it — 0.3% of a
debug build, zero on release. What it cannot answer, what incrementality
costs, wants a build run twice with the elapsed times compared. That is a
thing this census is shaped to do across many builds and no attribution of a
single run can replace it.
Measured, not assumed
hashdetail, 852 samples, redistributed into
lower/typeck/encodein thesame phases. 852 out, 852 in, nothing crossed a phase boundary. Summing
over
concerngives the phases back.emit(thewritesyscall) is matched innermost and could have takensamples from
asm. Inmachine_codeit took none — 18,901 before andafter. In
linkit took 11,860 offarchiveand left it 732, so it wasreverted there. A leaf marker is only safe where the leaf is all there is.
codegencarryingdebug_infomoved 4.3% of a debug build out of theleftover bucket without changing what its neighbours report.
Verification
Measured on clean release and debug builds of bun (99 workspace crates,
lto = "fat",codegen-units = 1): 730,142 and 394,791 samples, plus aself-build for the new column.
cargo test,cargo clippy --all-targets,cargo fmt --checkand the worker suite are clean.phases.parquetgrowsfrom ~8k to ~19k rows on a real build, still under half the span table.
🤖 Generated with Claude Code
https://claude.ai/code/session_015iY4ffQbZC8ix4g4fs1BR5