Skip to content

gh-161045: Fix tarfile writing stale pax fields of changed members - #161046

Open
gpshead wants to merge 2 commits into
python:mainfrom
gpshead:scanfix-tarfile-5iohylv3
Open

gpshead wants to merge 2 commits into
python:mainfrom
gpshead:scanfix-tarfile-5iohylv3

Conversation

@gpshead

@gpshead gpshead commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

TarInfo.create_pax_header() gave every record in pax_headers priority over the attributes, including records that were only there because the member was read from an archive, so renaming such a member (or changing its link name, owner, size or mtime) before addfile() was undone in the output. _apply_pax_info() now remembers each record it applied together with the attribute value it produced, and create_pax_header() drops a record that is still the one that was read when the attribute no longer has that value.

Records a program sets in pax_headers itself keep their priority (test_pax_extended_header is unchanged), and unchanged members are written as before, including high-precision mtime records. One side effect: a member whose name or link name came from a GNU long name or long link header preceding a pax path or linkpath record is now rewritten under the name tarfile reports for it.

A hdrcharset record kept from reading is no longer written back. It describes the header it was read from, and _create_pax_generic_header() adds hdrcharset=BINARY itself whenever a value needs it. Before, hdrcharset=BINARY could end up in front of values encoded in UTF-8: already on main for an unchanged member read with a non-UTF-8 encoding, and with the rest of this change also for a member renamed to a non-ASCII name.

This is meant for main only. Output changes for members whose attributes were changed, and TarInfo gains a _pax_applied slot, so a TarInfo pickled by a patched version would not load in an unpatched one of the same feature release.

Fixes #161045

ANT-2026-5IOHYLV3 ANT-2026-6CEC28KL

Fields of a pax extended header that were read from an archive no
longer have priority over TarInfo attributes that were changed
afterwards.  Fields set in TarInfo.pax_headers by the program and
fields of unchanged members are written as before.
@read-the-docs-community

read-the-docs-community Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #35825870 | 📁 Comparing 03927e3 against main (6b58f9a)

  🔍 Preview build  

150 files changed · ± 150 modified

± Modified

@gpshead
gpshead marked this pull request as ready for review November 1, 2026 03:18
@gpshead
gpshead requested a review from ethanfurman as a code owner November 1, 2026 03:18
@gpshead gpshead removed needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Nov 11, 2026
A hdrcharset=BINARY field kept in pax_headers from reading was written
in front of fields that create_pax_header() encodes in UTF-8, e.g. a
changed non-ASCII name.  _create_pax_generic_header() adds the field
itself whenever it is needed.

Also cover the link name in the GNU long header test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tarfile writes a member's old pax records after its name or other attributes are changed

1 participant