Skip to content

EditorState cursor model isn't grapheme-first — a redesign, not per-bug patches #374

Description

@bananabot9000

Summary

The EditorState cursor model was not designed grapheme-first. cursorCol is a code-unit index, and grapheme-awareness is bolted onto individual key handlers (left/right, recently backspace/delete) rather than enforced as an invariant. So any operation without a grapheme patch — most visibly char insert — can strand the cursor mid-cluster, and a later edit then splits the cluster into broken surrogates / dropped joiners.

Fixing these one at a time is whack-a-mole: each bug is one cockroach. Pulling up the carpet (the matrix below) shows it's a class, not a list — the fix is a grapheme-first cursor model, not more per-operation patches. This is a redesign, not a bug fix.

Symptom — the family (test matrix)

A fusing insert/paste (the inserted tail forms a cluster with the following char) leaves cursorCol mid-cluster. Ran against fix/ghost-text (dc2cfc4) — all fail:

insert before becomes a later delete yields
e combining acute é e (accent gone)
e combining grave è e
🇦 🇺 regional indicator 🇦🇺 flag half a flag (lone regional)
👍 🏽 skin-tone 👍🏽 👍 + lone modifier surrogate
👨 ‍👧 ZWJ+emoji 👨‍👧 family 👨👧 (ZWJ dropped)
️ VS16 ❤️ (variation selector dropped)
paste abce combining acute abcé cursor mid-é

Plus a separate vertical-movement defect: no goal/preferred column, so up/down through a short line permanently drifts the cursor to that line's width and never restores the original column.

import { describe, expect, it } from 'vitest';
import { EditorState } from '../src/model/EditorState.js';

const seg = new Intl.Segmenter(undefined, { granularity: 'grapheme' });
const onBoundary = (line: string, col: number): boolean =>
  col === 0 || col === line.length || [...seg.segment(line)].some((g) => g.index === col);

describe('EditorState cursor stays on a grapheme boundary', () => {
  it.each([
    { name: 'combining acute',     before: '́',          insert: 'e' },
    { name: 'combining grave',     before: '̀',          insert: 'e' },
    { name: 'regional flag',       before: '\u{1F1FA}',       insert: '\u{1F1E6}' },
    { name: 'skin-tone',           before: '\u{1F3FD}',       insert: '\u{1F44D}' },
    { name: 'ZWJ family',          before: '‍\u{1F467}', insert: '\u{1F468}' },
    { name: 'variation selector',  before: '️',          insert: '❤' },
    { name: 'paste fusing tail',   before: '́',          insert: 'abce' },
  ])('$name', ({ before, insert }) => {
    const s = new EditorState({ lines: [before], cursorLine: 0, cursorCol: 0 });
    s.handleKey({ type: 'char', value: insert });
    expect(onBoundary(s.lines[s.cursorLine] ?? '', s.cursorCol)).toBe(true);
  });
});

2,781 invariant checks confirm normal editing is currently fine — the failures are all at the fusing-insert and vertical edges.

Diagnosis

  • cursorCol is a code-unit offset. The invariant we actually want — the cursor is always on a grapheme boundary — is nowhere enforced; it's approximated by patching individual handlers.
  • char insert does cursorCol += value.length with no re-segmentation → a forward-fusing insert lands mid-cluster.
  • backspace/delete were patched to snap, but that can't rescue a cursor already stranded by the insert (delete slices slice(0, cursorCol) at the mid-cluster col regardless).
  • Every new operation needs its own grapheme patch → unbounded.

Proposal — grapheme-first

Make the boundary invariant structural, not tested after the fact:

  • Index the cursor by grapheme cluster, or normalize cursorCol to a boundary after every mutation, or route all cursor math through one grapheme-aware layer — so "cursor mid-cluster" becomes unrepresentable.
  • Done = the matrix above is green AND a property test (random op sequences over grapheme-rich text) holds "cursor on a boundary after every op" by construction.

Scope

Separate from #372 (the ghost-text render fix — a different subsystem; the cell-grid renderer is done and ships independently). This issue is the cursor model.

Detail and repros: the deep-hunt and matrix comments on #372.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions