Summary
The EditorState cursor model was not designed grapheme-first. cursorCol is a code-unit index, and grapheme-awareness is bolted onto individual key handlers (left/right, recently backspace/delete) rather than enforced as an invariant. So any operation without a grapheme patch — most visibly char insert — can strand the cursor mid-cluster, and a later edit then splits the cluster into broken surrogates / dropped joiners.
Fixing these one at a time is whack-a-mole: each bug is one cockroach. Pulling up the carpet (the matrix below) shows it's a class, not a list — the fix is a grapheme-first cursor model, not more per-operation patches. This is a redesign, not a bug fix.
Symptom — the family (test matrix)
A fusing insert/paste (the inserted tail forms a cluster with the following char) leaves cursorCol mid-cluster. Ran against fix/ghost-text (dc2cfc4) — all fail:
| insert |
before |
becomes |
a later delete yields |
e |
combining acute |
é |
e (accent gone) |
e |
combining grave |
è |
e |
| 🇦 |
🇺 regional indicator |
🇦🇺 flag |
half a flag (lone regional) |
| 👍 |
🏽 skin-tone |
👍🏽 |
👍 + lone modifier surrogate |
| 👨 |
👧 ZWJ+emoji |
👨👧 family |
👨👧 (ZWJ dropped) |
| ❤ |
️ VS16 |
❤️ |
❤ (variation selector dropped) |
paste abce |
combining acute |
abcé |
cursor mid-é |
Plus a separate vertical-movement defect: no goal/preferred column, so up/down through a short line permanently drifts the cursor to that line's width and never restores the original column.
import { describe, expect, it } from 'vitest';
import { EditorState } from '../src/model/EditorState.js';
const seg = new Intl.Segmenter(undefined, { granularity: 'grapheme' });
const onBoundary = (line: string, col: number): boolean =>
col === 0 || col === line.length || [...seg.segment(line)].some((g) => g.index === col);
describe('EditorState cursor stays on a grapheme boundary', () => {
it.each([
{ name: 'combining acute', before: '́', insert: 'e' },
{ name: 'combining grave', before: '̀', insert: 'e' },
{ name: 'regional flag', before: '\u{1F1FA}', insert: '\u{1F1E6}' },
{ name: 'skin-tone', before: '\u{1F3FD}', insert: '\u{1F44D}' },
{ name: 'ZWJ family', before: '\u{1F467}', insert: '\u{1F468}' },
{ name: 'variation selector', before: '️', insert: '❤' },
{ name: 'paste fusing tail', before: '́', insert: 'abce' },
])('$name', ({ before, insert }) => {
const s = new EditorState({ lines: [before], cursorLine: 0, cursorCol: 0 });
s.handleKey({ type: 'char', value: insert });
expect(onBoundary(s.lines[s.cursorLine] ?? '', s.cursorCol)).toBe(true);
});
});
2,781 invariant checks confirm normal editing is currently fine — the failures are all at the fusing-insert and vertical edges.
Diagnosis
cursorCol is a code-unit offset. The invariant we actually want — the cursor is always on a grapheme boundary — is nowhere enforced; it's approximated by patching individual handlers.
char insert does cursorCol += value.length with no re-segmentation → a forward-fusing insert lands mid-cluster.
backspace/delete were patched to snap, but that can't rescue a cursor already stranded by the insert (delete slices slice(0, cursorCol) at the mid-cluster col regardless).
- Every new operation needs its own grapheme patch → unbounded.
Proposal — grapheme-first
Make the boundary invariant structural, not tested after the fact:
- Index the cursor by grapheme cluster, or normalize
cursorCol to a boundary after every mutation, or route all cursor math through one grapheme-aware layer — so "cursor mid-cluster" becomes unrepresentable.
- Done = the matrix above is green AND a property test (random op sequences over grapheme-rich text) holds "cursor on a boundary after every op" by construction.
Scope
Separate from #372 (the ghost-text render fix — a different subsystem; the cell-grid renderer is done and ships independently). This issue is the cursor model.
Detail and repros: the deep-hunt and matrix comments on #372.
Summary
The
EditorStatecursor model was not designed grapheme-first.cursorColis a code-unit index, and grapheme-awareness is bolted onto individual key handlers (left/right, recentlybackspace/delete) rather than enforced as an invariant. So any operation without a grapheme patch — most visiblycharinsert — can strand the cursor mid-cluster, and a later edit then splits the cluster into broken surrogates / dropped joiners.Fixing these one at a time is whack-a-mole: each bug is one cockroach. Pulling up the carpet (the matrix below) shows it's a class, not a list — the fix is a grapheme-first cursor model, not more per-operation patches. This is a redesign, not a bug fix.
Symptom — the family (test matrix)
A fusing insert/paste (the inserted tail forms a cluster with the following char) leaves
cursorColmid-cluster. Ran againstfix/ghost-text(dc2cfc4) — all fail:deleteyieldseée(accent gone)eèe🇦🇺flag👍🏽👍+ lone modifier surrogate👨👧family👨👧(ZWJ dropped)❤️❤(variation selector dropped)abceabcééPlus a separate vertical-movement defect: no goal/preferred column, so
up/downthrough a short line permanently drifts the cursor to that line's width and never restores the original column.2,781invariant checks confirm normal editing is currently fine — the failures are all at the fusing-insert and vertical edges.Diagnosis
cursorColis a code-unit offset. The invariant we actually want — the cursor is always on a grapheme boundary — is nowhere enforced; it's approximated by patching individual handlers.charinsert doescursorCol += value.lengthwith no re-segmentation → a forward-fusing insert lands mid-cluster.backspace/deletewere patched to snap, but that can't rescue a cursor already stranded by the insert (deleteslicesslice(0, cursorCol)at the mid-cluster col regardless).Proposal — grapheme-first
Make the boundary invariant structural, not tested after the fact:
cursorColto a boundary after every mutation, or route all cursor math through one grapheme-aware layer — so "cursor mid-cluster" becomes unrepresentable.Scope
Separate from #372 (the ghost-text render fix — a different subsystem; the cell-grid renderer is done and ships independently). This issue is the cursor model.
Detail and repros: the deep-hunt and matrix comments on #372.