Skip to content

gh-153578: Fix out-of-bounds write in bytearray.extend() with a reentrant __buffer__ - #153579

Merged
serhiy-storchaka merged 2 commits into
python:mainfrom
tonghuaroot:extend-reentrant-buffer-oob
Aug 18, 2026
Merged

gh-153578: Fix out-of-bounds write in bytearray.extend() with a reentrant __buffer__#153579
serhiy-storchaka merged 2 commits into
python:mainfrom
tonghuaroot:extend-reentrant-buffer-oob

Conversation

@tonghuaroot

Copy link
Copy Markdown
Contributor

bytearray.extend() clamped only the high bound of the append range to the current size after acquiring the
argument's buffer, so a __buffer__ that shrinks the bytearray left the low bound past the high bound and ran
a negative-size memmove. Clamp the low bound too, matching bytearray.__iadd__.

… reentrant __buffer__

bytearray.extend() clamped only the high bound of the append range to the
current size after acquiring the argument's buffer, so a __buffer__ that shrinks
the bytearray left the low bound past the high bound and ran a negative-size
memmove. Clamp the low bound too, matching bytearray.__iadd__.
@serhiy-storchaka serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels Aug 17, 2026

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

Comment thread Lib/test/test_bytes.py
b = bytearray(b'x' * 100)
b.extend(Evil(lambda: b.__delitem__(slice(30, None))))
self.assertEqual(b, b'x' * 30 + b'ABCDEFGH')

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the third case:

        # grow during __buffer__: the data lands at the original end.
        b = bytearray(b'x' * 10)
        b.extend(Evil(lambda: b.extend(b'y' * 100)))
        self.assertEqual(b, b'x' * 10 + b'ABCDEFGH' + b'y' * 100)

@serhiy-storchaka

Copy link
Copy Markdown
Member

The defect isn't "reentrancy from a deliberately evil __buffer__" -- it is an ordinary data race that any GIL-releasing __buffer__ exposes. Please highlight in NEWS that this is a concurrency issue.

@tonghuaroot

Copy link
Copy Markdown
Contributor Author

Done in 5128c85: reframed the NEWS as a concurrency (any GIL-releasing __buffer__) out-of-bounds write, and added the grow-during-__buffer__ case.

@serhiy-storchaka
serhiy-storchaka merged commit e675e37 into python:main Aug 18, 2026
54 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @tonghuaroot for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

GH-156007 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Aug 18, 2026
@bedevere-app

bedevere-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

GH-156008 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Aug 18, 2026
@bedevere-app

bedevere-app Bot commented Aug 18, 2026

Copy link
Copy Markdown

GH-156009 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Aug 18, 2026
serhiy-storchaka pushed a commit that referenced this pull request Aug 18, 2026
…a reentrant __buffer__ (GH-153579) (GH-156008)

bytearray.extend() clamped only the high bound of the append range to the
current size after acquiring the argument's buffer, so a __buffer__ that shrinks
the bytearray left the low bound past the high bound and ran a negative-size
memmove. Clamp the low bound too, matching bytearray.__iadd__.
(cherry picked from commit e675e37)

Co-authored-by: tonghuaroot (童话) <tonghuaroot@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants