feat(fspy): let the caller size the tracking channel - #680
Merged
wan9chi merged 6 commits intoAug 18, 2026
Conversation
wan9chi
force-pushed
the
claude/fspy-shm-capacity-env
branch
from
August 17, 2026 06:23
5ca10c9 to
f7a2284
Compare
fspy benchmarklinuxmacoswindows |
wan9chi
force-pushed
the
claude/fspy-shm-capacity-env
branch
from
August 17, 2026 06:28
7bdf514 to
9eb5f30
Compare
The shared memory a tracked run reports its file accesses through was a constant in `fspy`, four gibibytes wide. How many accesses a program makes is the runner's business rather than the tracer's, and nothing could ask for a different size, so no test could put a task in front of a channel too small for it. `Command::shm_capacity` sets it, and the runner reads `VP_RUN_INTERNAL_FSPY_SHM_CAPACITY` for the value, keeping the same four gibibytes when the variable is unset. The variable is internal: it exists so a test can shrink the channel until a task overruns it, and nothing outside this repository should set it. A builder method rather than a second argument to `Command::new`, because the benchmark measures both revisions of `fspy` with a single launcher, overlaying the head's launcher source onto the baseline checkout. A launcher calling a signature only the head has cannot build the baseline arm. Leaving `new` alone also keeps the e2e tool, the examples and fspy's own tests out of this, since none of them care what size they get. The e2e case that comes with it stats one 2 MiB path, the largest single record tracking can be asked to hold, under a 64 MiB channel. That leaves room to spare, so the run caches like any other, which is what tells us the size arrived. The interesting case, a channel with no room for the record, has to wait: today it aborts the task process, and the panic it prints carries a thread id, a toolchain path, a backtrace and a platform's own abort code, none of which snapshot the same way twice. `vtt stat_long_filename` needed one fix to run there at all. Windows reports an over-long name as `ERROR_FILENAME_EXCED_RANGE`, which arrives as `InvalidFilename` rather than the `ENAMETOOLONG` unix returns, so the command exited 1 where it means to carry on: it exists to have the access attempted and recorded, not to find a file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Win32 spells `ERROR_FILENAME_EXCED_RANGE` without the second E, and the comment naming it is more use to a reader than the spelling checker is, so the word joins the allowed list beside the other Windows one. The `shm_capacity` field needed no musl exemption after all. It is read there, by the setter, so claiming it is dead made the expectation unfulfilled instead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Windows never overran the small channel the first version of this case set up, because a path record cannot get large enough there. A path reaches the tracer through a `UNICODE_STRING`, whose length field is a `u16`, so however long a name the caller asks for, no single record exceeds 64 KiB. Its 1 MiB channel had room to spare, tracking came back complete, and the run cached. Record count is the portable lever, and the slot table makes it exact: one slot per 64 bytes of the region, so a channel of a given size admits a known number of records whatever their paths look like. `vtt stat-many` makes as many accesses as asked for, under distinct names so none can fold into one record, and prints last to show the process outlived them. The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`4 << 30` says how the number is built; `4 * 1024 * 1024 * 1024` says what it is. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The size arrived through a builder on `fspy::Command`, a public default constant, and a `LazyLock` in the runner that read the override and passed it down. Three places to look, for a number with exactly one consumer. It now reads the override next to the `channel` call that uses it, and falls back to the default there. `Command` goes back to what it was, and so do the e2e tool, the examples, the benchmark launcher and fspy's own tests, none of which ever wanted a say in the size. The runner no longer names the variable at all, which also settles the musl question: `fspy::ipc` is already `cfg(not(target_env = "musl"))`, so the size lives behind the same gate as the channel it sizes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two commands stat generated names to be tracked; one varied the name's length and the other how many names. They are now `stat-many <count> [name-length]`, which also puts the name in kebab case with every other subcommand. Count leads because it is the knob that travels. A long name only fills a channel on unix: on Windows a path reaches the tracer through a `UNICODE_STRING` whose length is a `u16`, so no single record there exceeds 64 KiB however long a name the caller asks for. Names now carry their index, so a run of them cannot collapse into one record, and padding fills out whatever length is asked for. The `/dev/shm` case keeps its one 1 MiB name as `stat-many 1 1048576`, and gains the trailing line that reports the process survived its accesses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
wan9chi
force-pushed
the
claude/fspy-shm-capacity-env
branch
from
August 18, 2026 02:57
187994d to
17a5afc
Compare
wan9chi
added a commit
that referenced
this pull request
Aug 18, 2026
…ry publication (#675) ## Motivation Collecting a run's file accesses required every writer to hold still. The receiver could only read the frame stream once each sender had released a file lock, and that coupling broke in two ways: - [#544](#544): a traced process that closes inherited file descriptors (`python-daemon`, for one) releases the lock while keeping the shared memory writable. The reader then races live writers and reads path bytes as a frame header, panicking the runner. - A descendant that outlives the task blocks collection for as long as it runs. [#577](#577) tried an in-mapping writer count instead, but any process that dies between increment and decrement poisons it forever, since no userspace cleanup runs on `SIGKILL`. Correctness cannot depend on how long a file descriptor lives, or on writers running cleanup while the task tears down. ## The protocol `fspy_shared::ipc::channel::shm_io` publishes frames through a descriptor table instead of inline headers. `README.md` in that directory is the full description; the shape is: ```text | counters | descriptor table (one slot per frame) | payloads (the rest, grow up) | ``` - **Claiming is wait-free.** Two `fetch_add`s reserve payload bytes and a slot, with no retry loop and no lock. Each writer checks what the counter returned against a fixed limit, so overshooting costs nothing: no counter says where data is, since every committed descriptor carries its own offset and length. - **Publishing is one store.** The writer fills a span nobody else knows about, then stores the descriptor with `Release`. Only the writer that claimed a slot ever writes it, so it needs no compare-and-swap. The receiver loads with `Acquire`, so a descriptor it sees brings the payload bytes along. - **Death and abandonment are the same state.** An unfinished slot stays zero and the receiver ignores it. No cleanup code runs because none exists — no exit hooks, PID checks, heartbeats or timeouts. - **Sealing never waits and never writes.** One `swap` puts the CLOSED gate into the claim counter and reads the old value, drawing the boundary and shutting the gate at a single point in that counter's modification order. Sealing a channel holding ten million frames costs what sealing an empty one costs. Frames are then read in place, straight out of shared memory, with no copies. The channel asks one thing of its users: **publish a record before performing the action it describes.** A dead writer's missing record then describes an action that never happened, and a record refused after the seal describes one performed after the channel closed. The receiver drops both, and both answers are truthful. This holds across the Unix preload and the Windows detours; the Linux seccomp path collects supervisor-side and is unaffected. ## Running out of room A claim with nowhere to go sets the CLOSED gate before returning, and the writer skips that record and carries on — recording must never stop the program doing the work. The gate is also what tells the receiver: a seal that finds it already set hands back nothing, so the run is reported as untracked rather than as having touched only the paths that fit. Setting the bit first matters for the same reason publishing before acting does. If the seal misses the bit, the writer set it after the boundary, so the skipped record describes an action performed after the channel closed; and a writer that died before setting it never performed its action. This replaces a panic, and that panic is [#533](#533): on the base of this PR a full region aborts the traced process, because the panic fires inside an interposed `extern "C"` function that cannot unwind. It is reachable by workload rather than by any bug — #533 was hit as a SIGABRT storm when [vite-plus#2123](voidzero-dev/vite-plus#2123 `bunx` self-recursion flooded the channel — so a large enough build could kill the compiler doing the work. It now costs an uncached task instead, which is the fail-open behaviour that issue asks for, down to flagging the trace incomplete so nothing caches from it. One clause of #533 is not met, deliberately. It asks for no panic in a no-unwind context at all, and `Sender::send` still has asserts that only a disagreement between this crate and its codec can trip. Attaching a sender is a further, deliberate exception in the other direction: see the last point below. ## Consequences elsewhere - The lock file is gone. `ChannelConf` carries the shm id and the slot count, and `Receiver::lock` became a consuming, nonblocking `Receiver::close`. - The `ouroboros` self-referencing guard in `fspy::ipc` is gone with it, since frames are borrowed from the mapping the reader owns. - `Receiver::close` reports only the one failure a caller can act on — a record a sender could not write — and panics on a region that cannot hold the protocol, which `channel` proved it could before any sender saw it. - A run whose tracking came up short no longer fails the task. The runner reports it as a not-cached reason, since the task did its work and only the record of it is missing. - Attaching a sender now fails only when the channel is already over. Anything else — a file that will not open, or one that cannot hold the protocol — panics, because a process with no writer cannot tell the receiver it recorded nothing, and a trace that silently omits every access is worse than no trace. Panicking in the preload is not new: the base panics on a failed claim, an empty record, a size that does not fit a `usize`, and an underfilled frame. This PR removes the reachable one and adds one that only a broken channel reaches. Each file carries one argument: `layout` holds the shared shape, the descriptor codec and the three-rule memory-ordering contract that the code cites by rule number; `writer` and `reader` each carry a single aliasing justification. ## Known regression: Linux task launch The benchmark's Linux launch rows read **+158% dynamic and +221% static**, against roughly 0% on macOS and Windows and roughly 0% on every `access` row. Read that as an absolute number rather than a ratio: it is the one-off cost of first-touching the region's first pages, a millisecond or two, and the benchmark's launch target opens nothing at all, so a couple of milliseconds is most of what it measures. Against a real `tsc` or `vitest` task it is not visible. It is also per channel, which is per tracked spawn, so a build spawning hundreds of tasks pays it hundreds of times. Two things about it are settled by earlier work on this branch: the cost is in the fault path rather than block allocation, so `fallocate(KEEP_SIZE)` does not help (measured — it went backwards), and reads of holes on this runner cost the same as writes. A Linux-gated background pre-fault thread used to hide it and brought the row to ~+21-28%; it was removed on this branch because it buys nothing where `/tmp` is tmpfs and cost more than it saved on the other platforms. The fix that would remove the cost rather than mask it is to put the region on a filesystem that does not journal — `/dev/shm` on Linux, with `temp_dir()` as the fallback elsewhere. That is a separate change, and this project's history says to measure it rather than reason about it. Everything else measured flat: `access` +1.83%/+0.34% on Linux, +0.96% macOS, +0.17% Windows, all inside the benchmark's ~2.3pp noise floor. ## Verification Miri covers the protocol tests, including the slot state machine, seal races and concurrent writers. Cross-process tests cover real shared memory and a writer hard-killed mid-frame (`SIGKILL` / `TerminateProcess` via `Child::kill`). The e2e case from the base PR now does what it was written for. A 64 KiB channel holds a thousand records — one descriptor slot per 64 bytes of the region — and the task makes twenty thousand accesses, so the snapshot shows the task printing its last line and exiting cleanly, with the run reported as not cached, twice over: the second run does not replay an entry built from part of a trace. The case skips musl, which has no preload and collects through the seccomp supervisor, so there is no channel there to fill. Closes #544. Fixes #533. Partly addresses #605, whose signal-handler, post-`fork` and `errno` requirements this does not touch. Supersedes #577. Stacked on #680. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The shared memory a tracked run reports its file accesses through is four gibibytes, fixed. Nothing could ask for a smaller channel, so no test could put a task in front of one too small to hold its records — and that path decides whether a run may be cached. It had no coverage at all.
What changes
VP_RUN_INTERNAL_FSPY_SHM_CAPACITYoverrides the size, read next to thechannelcall that uses it and falling back to the same four gibibytes when unset. Nothing about a normal run moves. The variable is internal: it exists so a test can shrink the channel until a task overruns it, and nothing outside this repository should set it.The read sits at the point of use rather than travelling there. A size threaded through
Commandwould put a builder method, a public default constant and a lookup in the runner between the variable and its one consumer, and would drag every other caller — the benchmark launcher, the e2e tool, the examples, fspy's own tests — into a decision none of them want to make. It also keeps the whole thing behind onecfg:fspy::ipcis alreadycfg(not(target_env = "musl")), so the size lives behind the same gate as the channel it sizes.The test
vtt stat-many <count> [name-length]stats generated names to be tracked, under distinct names so none can fold into a single record, and prints its last line afterwards to show the process outlived them. It absorbsstat_long_filename, which did the same thing along the other axis; the/dev/shmcase that used it now saysstat-many 1 1048576. The e2e case makes twenty thousand accesses under a 64 MiB channel, which holds every one, so the run caches like any other — that is what tells us the size arrived where it was meant to.Record count is the lever rather than record size, because size cannot be pushed far enough on every platform. A path reaches the tracer on Windows through a
UNICODE_STRING, whose length field is au16, so no single record there exceeds 64 KiB however long a name the caller asks for — an earlier version of this case tried one 2 MiB path and Windows had room to spare for it.The case skips musl, which has no preload: those builds collect through the seccomp supervisor, on the runner's own side of the boundary, so there is no shared-memory channel there to fill.
The case worth testing, a channel too small for the task, has to wait for #675. On this base a full channel aborts the task process, and the panic it prints carries a thread id, a toolchain path, a backtrace and a platform's own abort code, so there is nothing there that snapshots the same way twice. #675 makes that a skipped record and a reported reason instead, and updates this snapshot to show it.
Split out of #675.
🤖 Generated with Claude Code