Skip to content

gh-162246: Fix lost writes on BufferedRWPair dealloc - #162295

Merged
cmaloney merged 7 commits into
python:mainfrom
maurycy:gh-162246-bufferedrwpair-data-loss
Nov 21, 2026
Merged

cmaloney merged 7 commits into
python:mainfrom
maurycy:gh-162246-bufferedrwpair-data-loss

Conversation

@maurycy

@maurycy maurycy commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

socket.makefile("rwb") passes the same raw stream to both sides of a BufferedRWPair. When the pair is deallocated, the reader closes the stream before the writer can flush its buffer.

Release the writer first, matching the order in BufferedRWPair.close().

Comment thread Lib/test/test_io/test_bufferedio.py Resolved
@cmaloney cmaloney added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Nov 19, 2026

@cmaloney cmaloney left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this ordering works better. May be worthwhile to add a comment that the Py_CLEAR order is important in this case.

@cmaloney
cmaloney merged commit dc222c8 into python:main Nov 21, 2026
104 of 105 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @maurycy for the PR, and @cmaloney for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @maurycy and @cmaloney, I could not cleanly backport this to 3.15 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker dc222c8d6c366e3d52d4b0d0053ad7bb8a601fff 3.15

@miss-islington-app

Copy link
Copy Markdown

Sorry, @maurycy and @cmaloney, I could not cleanly backport this to 3.14 due to a conflict.

Please backport manually with cherry_picker, see the devguide for more information.

cherry_picker dc222c8d6c366e3d52d4b0d0053ad7bb8a601fff 3.14

@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 MacOS Intel ASAN NoGIL 3.x (tier-2) has failed when building commit dc222c8.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1366/builds/8615) and take a look at the build logs.
  4. Check if the failure is related to this commit (dc222c8) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1366/builds/8615

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/support/__init__.py", line 1004, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.BufferedReader name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example-21.12-py3-none-any.whl'>
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example-21.12-py3.6.egg' mode='rb' closefd=True>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/support/__init__.py", line 1004, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.BufferedReader name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example-21.12-py3.6.egg'>
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example2-1.0.0-py3-none-any.whl' mode='rb' closefd=True>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/support/__init__.py", line 1004, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.BufferedReader name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example2-1.0.0-py3-none-any.whl'>
k


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/support/__init__.py", line 1004, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.BufferedReader name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example-21.12-py3.6.egg'>
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example-21.12-py3-none-any.whl' mode='rb' closefd=True>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/support/__init__.py", line 1004, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.BufferedReader name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example-21.12-py3-none-any.whl'>
Warning -- Unraisable exception
Exception ignored while finalizing file <_io.FileIO name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example2-1.0.0-py3-none-any.whl' mode='rb' closefd=True>:
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/support/__init__.py", line 1004, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.BufferedReader name='/Users/buildbot/buildarea/3.x.itamaro-macos-intel-aws.macos-with-brew.asan.nogil/build/Lib/test/test_importlib/metadata/data/example2-1.0.0-py3-none-any.whl'>
k

@bedevere-app

bedevere-app Bot commented Nov 21, 2026

Copy link
Copy Markdown

GH-162344 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Nov 21, 2026
@bedevere-app

bedevere-app Bot commented Nov 21, 2026

Copy link
Copy Markdown

GH-162345 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 21, 2026
cmaloney added a commit that referenced this pull request Nov 21, 2026
) (#162345)

`socket.makefile("rwb")` passes the same raw stream to both sides of a `BufferedRWPair`. When the pair is deallocated, the reader closes the stream before the writer can flush its buffer.

Release the writer first, matching the order in `BufferedRWPair.close()`.
(cherry picked from commit dc222c8)

Co-authored-by: Maurycy Pawłowski-Wieroński <maurycy@maurycy.com>
@bedevere-app

bedevere-app Bot commented Nov 21, 2026

Copy link
Copy Markdown

GH-162351 is a backport of this pull request to the 3.15 branch.

@maurycy
maurycy deleted the gh-162246-bufferedrwpair-data-loss branch November 21, 2026 04:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants