fix(rum): stop attributing evaluated-code stacks to Studio - #1719
Merged
dawsontoth merged 1 commit intoSep 18, 2026
Merged
Conversation
Coverage Report
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||
Contributor
There was a problem hiding this comment.
Code Review
This pull request addresses issue #1659 by filtering out error events where the stack trace consists entirely of evaluated code frames (e.g., :line:col). It introduces a new regular expression EVALUATED_FRAME to identify these frames and updates the originatesInThirdPartyScript helper to classify them as foreign. Comprehensive regression tests are also added to verify various stack trace scenarios. The feedback suggests making the wildcard match in the EVALUATED_FRAME regular expression non-greedy to prevent potential catastrophic backtracking on long lines.
dawsontoth
force-pushed
the
claude/rum-drop-unattributable-extension-stacks
branch
3 times, most recently
from
September 18, 2026 14:45
2b65b7f to
7273e88
Compare
An anti-fingerprinting extension wraps WebGLRenderingContext.getParameter through eval and recurses into its own wrapper, so every frame it contributes carries a position inside evaluated code and no script URL. FRAME_URL matches neither, so each line was skipped as unlocatable, sawThirdPartyFrame stayed false, and the RangeError was kept as Studio's own. Treat a frame whose location is a position inside evaluated code as evidence of foreign code. Studio ships from /assets/*.js and never evaluates at runtime, so such a frame is never ours. The line:column is required, which is what keeps #1646's contract intact: a bare `at <anonymous>` stays unattributable and neither side claims it. Closes #1659 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dawsontoth
force-pushed
the
claude/rum-drop-unattributable-extension-stacks
branch
from
September 18, 2026 14:57
7273e88 to
1d90cd1
Compare
dawsontoth
marked this pull request as ready for review
September 18, 2026 15:35
kriszyp
approved these changes
Sep 18, 2026
dawsontoth
deleted the
claude/rum-drop-unattributable-extension-stacks
branch
September 18, 2026 23:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1659.
An anti-fingerprinting browser extension wraps
WebGLRenderingContext.getParameterthroughevaland recurses into its own wrapper. Every frame it contributes carries a position inside evaluated code and no script URL at all:FRAME_URLmatches only a frame ending in a realhttps?://,blob:or*-extension://URL, so every line was skipped as unlocatable,sawThirdPartyFramestayedfalse, and theRangeErrorwas kept and attributed to Studio. Studio uses no WebGL anywhere insrc/.This teaches the attribution one more thing that is provably not ours: a frame positioned inside evaluated code. Studio ships entirely from
/assets/*.jsand never evaluates code at runtime.What this deliberately does not do
#1659 left the design open, and the first framing review returned
better-alternative-existsagainst the broad reading ("treat any stack with zero located frames as third-party"). This implements the narrow alternative both the reviewer and the existing tests point to:[native code],undefined, and empty locations stay neutral. They say nothing about who is on the stack. An opaque cross-originUncaught "Script error."is still kept.at <anonymous>with no line:column stays neutral, because that is what the browser emits when it knows nothing at all — Stop reporting browser-extension errors to Datadog RUM #1646 already settled that an unresolvable frame is skipped rather than attributed.For the human reviewer
The framing verdict cleared only in the narrow form above; the ledger below is the adjudicator's open judgement calls, answered.
Evaluated frames as evidence of foreign origin. The premise is "Studio never evals". If a bundled dependency ever adopted
new Function, its evaluated frame would be classed foreign — but that stack would also carry the dependency's own/assets/*.jscaller frame, which short-circuits to keep. The exposure is limited to stacks that are entirely evaluated code, which by construction have no first-party frame. Reversible by deleting one regex; events dropped in the interim are not recoverable.Message text versus frames — fixed, not accepted. The adjudicator caught that a multi-line message puts server- or customer-composed text on its own line of the serialized stack, and Harper builds that text by interpolation, so it can be shaped exactly like a frame (
redactRelayedMessage.tsdocuments the same hazard).shouldKeepEventruns on the raw stack before redaction, so redaction could not save it. The scan now skips the message header span, reusing the computationredactRelayedStackalready had — extracted asmessageHeaderLengthrather than duplicated. This also closes the same hole for the pre-existing extension-URL path, which had it before this change.Eval token breadth — resolved rather than left open. An earlier revision matched
eval[^\s]*, which would also match unrelatedeval-prefixed tokens. V8 gives anevalframe's own position an<anonymous>spelling (at eval (eval at <anonymous> (…), <anonymous>:1:7)), and 30 days of RUM contain zero frames carrying anevaltoken, so the exact<anonymous>location covers it with less surface.When neither header spelling opens the stack, the scan starts at zero — i.e. it reads the whole stack, message included, exactly as it did before this change. That is the conservative direction for a filter whose failure mode is dropping real errors: an unrecognised header costs the extra protection, never more than the status quo.
Raw V8
at <anonymous>:1:7frames are not matched, becauseEVALUATED_FRAMErequires the[\s(@]separator. The final round raised and then disputed this itself:toStackTraceStringalways emitsat <func> @ <url>, substituting a literal<anonymous>for an unknown function, so the separator is on every stackerror.stackcarries — the same assumptionFRAME_URLalready ships on. All 335 URL-less frames in the 30-day corpus carry it. Recorded as a robustness gap, not a live false-keep.Cursor-composer's findings in round 1 cited
DatabaseTableView.tsx,deleteTableRecords.tsandTableView.test.tsx— none of which are in this diff. That lens reviewed a different change and its findings are not carried. From round 3 on it dropped out entirely, which is structural rather than a failure: this PR editsAGENTS.md, and the Cursor legs refuse a diff that changes agent instructions. Independent coverage across the five rounds is codex + gemini + Harper adjudication, which converged at round 4.Verification
End-to-end route: replay of the real production corpus, since this filter's whole contract is about stacks only production produces.
error.id, old module vs new): 29 events dropped, all of them the WebGLRangeErrorfamily; zero other events changed verdict in either direction. Re-run after the message-header fix with the identical result, which is what shows that skipping the header does not readmit any previously-dropped extension noise. Reviewers cannot re-run this — the evidence is off-repo, pulled withpup rum events.at-prefix contract this relies on is empirically confirmed, not assumed: the same corpus contains 217 Safari / Mobile Safari error events and zero frames in Safari'sfunc@urlform, so the SDK'stoStackTraceStringnormalization holds across engines.evalposition, Studio frames in both orders, the Datadog instrumentation wrapper, native-only and positionless<anonymous>frames, and a message-borne evaluated location with no frame to support it. Added tobeforeSend.test.tsas well, so the composition with redaction is proven, not just the helper.||=to&&=, dropping the line:column requirement, weakening the frame anchor, and removing the Studio short-circuit each turn tests red. Two gaps found this way (positionless@ <anonymous>, and an embedded location rescued by a Studio frame rather than by the anchor) are closed by two tests added for exactly that reason.tsc -b,oxlint,dprint checkall exit 0.Complexity: S
Review-Coverage: authored=claude; ran=gemini,cursor-composer,codex; adjudicated=domain; declined=cursor-grok; rounds=5; full=3 @ 1d90cd1
Human-Review-Need: 3 (decisions: evaluated-frame-as-foreign, share-header-length-with-redaction, unverified-header-scans-from-zero, no-end-to-end-normalization-coverage) @ 1d90cd1