Skip to content

GH-51669: [C++][Parquet] Preserve file algorithm in plaintext footers - #51693

Open
YusefSyed wants to merge 1 commit into
apache:mainfrom
YusefSyed:codex/gh-51669-plaintext-footer-algorithm
Open

YusefSyed wants to merge 1 commit into
apache:mainfrom
YusefSyed:codex/gh-51669-plaintext-footer-algorithm

Conversation

@YusefSyed

@YusefSyed YusefSyed commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Fixes #51669. With AES_GCM_CTR_V1 and a plaintext footer, the writer encrypts data pages with CTR but records AES_GCM_V1 in FileMetaData.encryption_algorithm. A reader consequently uses the wrong file algorithm and cannot read an otherwise successful write.

What changes are included in this PR?

Record the configured file algorithm in plaintext-footer metadata. Footer signing and verification continue to use the existing metadata cipher path. Add native checks for the recorded algorithm, synchronous/asynchronous reads, wrong-key rejection, and tampered footer signatures; extend the Python direct-key test across both algorithms and footer modes.

Are these changes tested?

  • Compiled the unchanged native baseline with the new regression: it failed on the CTR algorithm metadata and data read.
  • Compiled the correction: the native encryption suite passed 34 tests, including existing encrypted-footer and AAD controls. The focused regression also passed after the final comment clarification.
  • Verified the test executable loads the locally built Arrow and Parquet libraries.
  • C++ formatting, repository-configured Python lint, and git diff --check passed.
  • Built PyArrow from the contribution source (26.0.0a1.dev2+g84f7d3fec), linked to the task-built native libraries: 22 direct-key tests passed, and the standalone algorithm/footer matrix passed 4/4.
  • Additional plaintext-footer checks passed for both ciphers with stored/external AAD prefixes, including wrong-key, wrong-AAD, and tampered-footer rejection.
  • Re-ran the full native encryption suite after completing the native build configuration: 34/34 passed. Source hashes, build logs, and loaded-library paths establish the tested source; the native build-info Git ID remains its earlier configure-time stamp.

Native test command, after building parquet-encryption-test with encryption enabled:

PARQUET_TEST_DATA=/path/to/parquet-testing/data /path/to/build/release/parquet-encryption-test

Validation so far is on macOS arm64. No Windows/Linux local result or live KMS result is claimed.

Are there any user-facing changes?

New plaintext-footer files written with AES_GCM_CTR_V1 carry the correct algorithm metadata. Existing malformed files are not repaired automatically. Public APIs are unchanged.

This PR contains a "Critical Fix" under the template's invalid-data criterion: affected writes produced inconsistent encryption metadata and unreadable files. This is not a claim of a newly demonstrated security vulnerability.

Was AI used for this PR?

PR code and description written by:

  • Human
  • AI

Reviewed before submission by:

  • Human
  • AI
  • Not reviewed

Codex assisted with investigation, implementation, regression tests, and this description. The main Codex agent reviewed the production/test diff, the file-algorithm versus metadata-cipher distinction, and the recorded native test results. No human review or test execution is asserted on the submitter's behalf.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #51669 has been automatically assigned in GitHub to PR creator.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #51669 has no components, please add labels for components.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #51669 has been automatically assigned in GitHub to PR creator.

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

⚠️ GitHub issue #51669 has no components, please add labels for components.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[C++][Parquet] Plaintext-footer files written with AES_GCM_CTR_V1 record AES_GCM_V1 as the encryption algorithm and cannot be read

1 participant