Skip to content

fix(rum): stop attributing evaluated-code stacks to Studio - #1719

Merged
dawsontoth merged 1 commit into
stagefrom
claude/rum-drop-unattributable-extension-stacks
Sep 18, 2026
Merged

dawsontoth merged 1 commit into
stagefrom
claude/rum-drop-unattributable-extension-stacks

Conversation

@dawsontoth

@dawsontoth dawsontoth commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Closes #1659.

An anti-fingerprinting browser extension wraps WebGLRenderingContext.getParameter through eval and recurses into its own wrapper. Every frame it contributes carries a position inside evaluated code and no script URL at all:

RangeError: Maximum call stack size exceeded
  at WebGLRenderingContext.value [as getParameter] @ <anonymous>:4:28
  at WebGLRenderingContext.value [as getParameter] @ <anonymous>:7:56
  … repeating

FRAME_URL matches only a frame ending in a real https?://, blob: or *-extension:// URL, so every line was skipped as unlocatable, sawThirdPartyFrame stayed false, and the RangeError was kept and attributed to Studio. Studio uses no WebGL anywhere in src/.

This teaches the attribution one more thing that is provably not ours: a frame positioned inside evaluated code. Studio ships entirely from /assets/*.js and never evaluates code at runtime.

What this deliberately does not do

#1659 left the design open, and the first framing review returned better-alternative-exists against the broad reading ("treat any stack with zero located frames as third-party"). This implements the narrow alternative both the reviewer and the existing tests point to:

For the human reviewer

The framing verdict cleared only in the narrow form above; the ledger below is the adjudicator's open judgement calls, answered.

  • Evaluated frames as evidence of foreign origin. The premise is "Studio never evals". If a bundled dependency ever adopted new Function, its evaluated frame would be classed foreign — but that stack would also carry the dependency's own /assets/*.js caller frame, which short-circuits to keep. The exposure is limited to stacks that are entirely evaluated code, which by construction have no first-party frame. Reversible by deleting one regex; events dropped in the interim are not recoverable.

  • Message text versus frames — fixed, not accepted. The adjudicator caught that a multi-line message puts server- or customer-composed text on its own line of the serialized stack, and Harper builds that text by interpolation, so it can be shaped exactly like a frame (redactRelayedMessage.ts documents the same hazard). shouldKeepEvent runs on the raw stack before redaction, so redaction could not save it. The scan now skips the message header span, reusing the computation redactRelayedStack already had — extracted as messageHeaderLength rather than duplicated. This also closes the same hole for the pre-existing extension-URL path, which had it before this change.

  • Eval token breadth — resolved rather than left open. An earlier revision matched eval[^\s]*, which would also match unrelated eval-prefixed tokens. V8 gives an eval frame's own position an <anonymous> spelling (at eval (eval at <anonymous> (…), <anonymous>:1:7)), and 30 days of RUM contain zero frames carrying an eval token, so the exact <anonymous> location covers it with less surface.

  • When neither header spelling opens the stack, the scan starts at zero — i.e. it reads the whole stack, message included, exactly as it did before this change. That is the conservative direction for a filter whose failure mode is dropping real errors: an unrecognised header costs the extra protection, never more than the status quo.

  • Raw V8 at <anonymous>:1:7 frames are not matched, because EVALUATED_FRAME requires the [\s(@] separator. The final round raised and then disputed this itself: toStackTraceString always emits at <func> @ <url>, substituting a literal <anonymous> for an unknown function, so the separator is on every stack error.stack carries — the same assumption FRAME_URL already ships on. All 335 URL-less frames in the 30-day corpus carry it. Recorded as a robustness gap, not a live false-keep.

Cursor-composer's findings in round 1 cited DatabaseTableView.tsx, deleteTableRecords.ts and TableView.test.tsx — none of which are in this diff. That lens reviewed a different change and its findings are not carried. From round 3 on it dropped out entirely, which is structural rather than a failure: this PR edits AGENTS.md, and the Cursor legs refuse a diff that changes agent instructions. Independent coverage across the five rounds is codex + gemini + Harper adjudication, which converged at round 4.

Verification

End-to-end route: replay of the real production corpus, since this filter's whole contract is about stacks only production produces.

  • 30 days of RUM replayed through the actual predicate (2,000 events deduplicated by error.id, old module vs new): 29 events dropped, all of them the WebGL RangeError family; zero other events changed verdict in either direction. Re-run after the message-header fix with the identical result, which is what shows that skipping the header does not readmit any previously-dropped extension noise. Reviewers cannot re-run this — the evidence is off-repo, pulled with pup rum events.
  • The at -prefix contract this relies on is empirically confirmed, not assumed: the same corpus contains 217 Safari / Mobile Safari error events and zero frames in Safari's func@url form, so the SDK's toStackTraceString normalization holds across engines.
  • Unit tests cover the production WebGL stack, its CRLF variant, a real V8 eval position, Studio frames in both orders, the Datadog instrumentation wrapper, native-only and positionless <anonymous> frames, and a message-borne evaluated location with no frame to support it. Added to beforeSend.test.ts as well, so the composition with redaction is proven, not just the helper.
  • Every test was mutation-checked. Removing the evidence line, flipping its ||= to &&=, dropping the line:column requirement, weakening the frame anchor, and removing the Studio short-circuit each turn tests red. Two gaps found this way (positionless @ <anonymous>, and an embedded location rescued by a Studio frame rather than by the anchor) are closed by two tests added for exactly that reason.
  • Full gate green on the final commit: vitest 3,156 passed / 11 skipped, tsc -b, oxlint, dprint check all exit 0.

Complexity: S

Review-Coverage: authored=claude; ran=gemini,cursor-composer,codex; adjudicated=domain; declined=cursor-grok; rounds=5; full=3 @ 1d90cd1

Human-Review-Need: 3 (decisions: evaluated-frame-as-foreign, share-header-length-with-redaction, unverified-header-scans-from-zero, no-end-to-end-normalization-coverage) @ 1d90cd1

@github-actions

github-actions Bot commented Sep 18, 2026

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 63.24% 8892 / 14059
🔵 Statements 63.57% 9496 / 14937
🔵 Functions 55.83% 2228 / 3990
🔵 Branches 57.22% 6477 / 11318
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/integrations/datadog/redactRelayedMessage.ts 93.75% 92.85% 100% 93.75% 66
src/integrations/datadog/shouldKeepEvent.ts 100% 100% 100% 100%
Generated in workflow #1930 for commit 1d90cd1 by the Vitest Coverage Report Action

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses issue #1659 by filtering out error events where the stack trace consists entirely of evaluated code frames (e.g., :line:col). It introduces a new regular expression EVALUATED_FRAME to identify these frames and updates the originatesInThirdPartyScript helper to classify them as foreign. Comprehensive regression tests are also added to verify various stack trace scenarios. The feedback suggests making the wildcard match in the EVALUATED_FRAME regular expression non-greedy to prevent potential catastrophic backtracking on long lines.

Comment thread src/integrations/datadog/shouldKeepEvent.ts
@dawsontoth
dawsontoth force-pushed the claude/rum-drop-unattributable-extension-stacks branch 3 times, most recently from 2b65b7f to 7273e88 Compare September 18, 2026 14:45
An anti-fingerprinting extension wraps WebGLRenderingContext.getParameter through eval and
recurses into its own wrapper, so every frame it contributes carries a position inside
evaluated code and no script URL. FRAME_URL matches neither, so each line was skipped as
unlocatable, sawThirdPartyFrame stayed false, and the RangeError was kept as Studio's own.

Treat a frame whose location is a position inside evaluated code as evidence of foreign
code. Studio ships from /assets/*.js and never evaluates at runtime, so such a frame is
never ours. The line:column is required, which is what keeps #1646's contract intact: a
bare `at <anonymous>` stays unattributable and neither side claims it.

Closes #1659

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dawsontoth
dawsontoth force-pushed the claude/rum-drop-unattributable-extension-stacks branch from 7273e88 to 1d90cd1 Compare September 18, 2026 14:57
@dawsontoth
dawsontoth marked this pull request as ready for review September 18, 2026 15:35
@dawsontoth
dawsontoth requested a review from a team as a code owner September 18, 2026 15:35

@cb1kenobi cb1kenobi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No confirmed blocking issues were found in the changed lines. The attribution change handles evaluated frames while preserving first-party and unlocatable-stack behavior.


Reviewed 1d90cd1

@dawsontoth
dawsontoth added this pull request to the merge queue Sep 18, 2026

@cb1kenobi cb1kenobi left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No confirmed blocking issues were found on changed lines. Existing discussion already covers the non-blocking regex concern.


Reviewed 1d90cd1

Merged via the queue into stage with commit c28b423 Sep 18, 2026
5 checks passed
@dawsontoth
dawsontoth deleted the claude/rum-drop-unattributable-extension-stacks branch September 18, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[RUM] Extension WebGL fingerprinting RangeError reaches Error Tracking — the stack filter skips frames with no URL

3 participants