Skip to content

gh-152845: Preserve ZIP encoding flags when appending - #161950

Merged
ambv merged 2 commits into
python:mainfrom
KasraGhodsi:zipfile-preserve-encoding-flags
Nov 13, 2026
Merged

ambv merged 2 commits into
python:mainfrom
KasraGhodsi:zipfile-preserve-encoding-flags

Conversation

@KasraGhodsi

Copy link
Copy Markdown
Contributor

Summary

Preserve the UTF-8/EFS flag when zipfile rewrites existing entries in append mode, including ASCII filenames with UTF-8 entry comments. This keeps the central directory consistent with retained local headers and preserves the encoding indicated for comment bytes.

Keep existing new-entry defaults: ordinary ASCII names remain CP437-tagged, explicit UTF-8 flags are honored, and new non-ASCII names remain UTF-8-tagged. Comments remain opaque bytes; the implementation does not infer UTF-8 merely because comment bytes are non-ASCII.

This is a narrowly scoped alternative to #152846 for the flag-preservation portion of #152845. For example, b'\x82' is a valid CP437 comment but not valid UTF-8; changing its encoding flag without transcoding mislabels the data. The regression coverage also checks explicitly flagged ASCII entries with empty or ASCII comments. This PR does not add metadata_encoding support to writing modes or additional encoding state to ZipInfo.

Testing

Tested on Linux aarch64 with a debug CPython 3.16.0a2+dev build, GCC 11.5.0, based on 727b56b4db5734864953a6586ea03633ef27bad5.

  • The unpatched baseline reproduced the retained-entry and explicit-UTF-8 flag-loss regressions; the fixed append regression and five-case new-entry test pass.
  • ./python -m test -v test_zipfile completed successfully: 607 tests, three skips.
  • ./python -m test -j4 completed successfully: 51,364 tests reported run, 2,963 skipped.
  • Patchcheck completed using explicit local upstream/main for default-branch discovery; the standard invocation crashes in that Git lookup.
  • git diff --check passed.

Related work and scope

Issue: gh-152845

Related alternative implementation: #152846. This PR keeps the append bug fix separate from broader comment-encoding policy and metadata_encoding enhancements; affected maintenance-branch backports can be considered separately.

@python-cla-bot

python-cla-bot Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

@ambv
ambv merged commit 5151e7f into python:main Nov 13, 2026
57 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @KasraGhodsi for the PR, and @ambv for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Nov 13, 2026

Copy link
Copy Markdown

GH-161960 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Nov 13, 2026
@bedevere-app

bedevere-app Bot commented Nov 13, 2026

Copy link
Copy Markdown

GH-161961 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Nov 13, 2026
ambv pushed a commit that referenced this pull request Nov 14, 2026
…) (gh-161961)

(cherry picked from commit 5151e7f)

Co-authored-by: Kasra Ghodsi <kasra.ghodsi@gmail.com>
ambv pushed a commit that referenced this pull request Nov 14, 2026
…) (gh-161960)

(cherry picked from commit 5151e7f)

Co-authored-by: Kasra Ghodsi <kasra.ghodsi@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants