Skip to content

fix(workflows): report overlay operation keys in declaration order - #4146

Open
jawwad-ali wants to merge 1 commit into
github:mainfrom
jawwad-ali:fix/overlay-shorthand-key-order
Open

fix(workflows): report overlay operation keys in declaration order#4146
jawwad-ali wants to merge 1 commit into
github:mainfrom
jawwad-ali:fix/overlay-shorthand-key-order

Conversation

@jawwad-ali

Copy link
Copy Markdown
Contributor

Problem

_parse_edit collects the shorthand operation keys by iterating the frozenset:

shorthand_keys = [key for key in _SHORTHAND_OPERATION_KEYS if key in edit_raw]

_SHORTHAND_OPERATION_KEYS is VALID_OPERATIONS, a frozenset. Its iteration order for these strings depends on per-process string-hash randomization, so the two error messages built from that list name the offending keys in a different order on every run for the exact same overlay file.

Reproduction on current main

Six consecutive fresh processes, same input:

["Edit at index 0 has multiple operation keys: 'insert_after', 'remove'."]
["Edit at index 0 has multiple operation keys: 'remove', 'insert_after'."]
["Edit at index 0 has multiple operation keys: 'remove', 'insert_after'."]
["Edit at index 0 has multiple operation keys: 'remove', 'insert_after'."]
["Edit at index 0 has multiple operation keys: 'insert_after', 'remove'."]
["Edit at index 0 has multiple operation keys: 'remove', 'insert_after'."]

Underlying cause, same six processes:

['insert_before', 'insert_after', 'remove', 'replace']
['insert_after', 'insert_before', 'remove', 'replace']
['remove', 'replace', 'insert_before', 'insert_after']
['insert_before', 'insert_after', 'remove', 'replace']
['insert_after', 'insert_before', 'remove', 'replace']
['remove', 'replace', 'insert_before', 'insert_after']

Both messages are affected — the multiple-keys one above, and "...mixes shorthand operation key ({shorthand_keys[0]!r})...", which picks an arbitrary one of the user's keys to name.

This is reached by a very ordinary YAML slip: forgetting the - on a second edit merges both operation keys into one mapping.

Fix

Iterate edit_raw, which yields the user's declared order and is deterministic. Dict keys are always hashable, so the membership test is safe in this direction too.

Verification

The test is designed to fail deterministically in every process, not ~50% of the time: it parametrizes both declaration orders. Whatever fixed order a frozenset happens to have in a given process, one of the two parametrizations must contradict it.

  • Fail-before / pass-after: 2 new-vs-baseline failures with the source reverted → 37 passed with the fix. (In the recording process the frozenset ordered insert_after first, so the two remove-first cases were the ones that failed — in a process with the opposite order, the insert_after-first pair fails instead.)
  • Scoped regression over tests/workflows: no new failures vs a clean-main baseline (10 pre-existing, Windows symlink-privilege).
  • uvx ruff@0.15.0 check src tests → clean

No breaking change — message content is unchanged, only its ordering becomes stable. The existing test_multiple_operation_fields_rejected (which asserts any("multiple" in e.lower() ...)) still passes untouched.


Written with assistance from Claude Code. Bug found, reproduced, and verified by me on current main.

`_parse_edit` collected the shorthand operation keys by iterating the
frozenset:

    shorthand_keys = [key for key in _SHORTHAND_OPERATION_KEYS if key in edit_raw]

`_SHORTHAND_OPERATION_KEYS` is `VALID_OPERATIONS`, a frozenset, so its
iteration order depends on per-process string-hash randomization. The two
error messages built from that list named the offending keys in a different
order on every run for the exact same overlay file:

  ["Edit at index 0 has multiple operation keys: 'insert_after', 'remove'."]
  ["Edit at index 0 has multiple operation keys: 'remove', 'insert_after'."]
  ["Edit at index 0 has multiple operation keys: 'remove', 'insert_after'."]

Iterating `edit_raw` instead yields the user's declared order and is
deterministic. Dict keys are always hashable, so the membership test is
safe in this direction too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jawwad-ali
jawwad-ali requested a review from mnriem as a code owner August 15, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant