Skip to content

fix(spec): validate decimal mantissas before conversion - #3655

Open
slfan1989 wants to merge 1 commit into
apache:mainfrom
slfan1989:iceberg-rust-3654
Open

slfan1989 wants to merge 1 commit into
apache:mainfrom
slfan1989:iceberg-rust-3654

Conversation

@slfan1989

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

Apply positive exponents before extracting the unscaled decimal value, so 1e3 produces 1000. Validate precision before converting the coefficient to i128, so oversized inputs return DataInvalid instead of silently producing an incorrect value. JSON decimal parsing also validates the declared precision after rounding to the target scale.

Are these changes tested?

Added three regression tests covering positive exponents, the 38-digit precision boundary, and precision validation at the target JSON scale. All three failed before the fix and passed after it.

AI Disclosure

Copilot AI balanced review requested due to automatic review settings November 7, 2026 09:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the decimal parsing and precision-validation changes.

No open issues 🎉

What changed in this PR

Fixes decimal parsing in Iceberg’s value layer to address #3654: positive exponents are preserved, and oversized coefficients are rejected before conversion to i128.

Changes:

  • Centralizes checked decimal coefficient extraction with scale and precision validation.
  • Validates JSON decimal precision after rounding to the target scale.
  • Adds regression coverage for exponent handling, oversized values, and JSON scale conversion.
File Description
crates/​iceberg/​src/​writer/​file_writer/​parquet_writer.rs Updates test calls for fallible coefficient extraction.
crates/​iceberg/​src/​spec/​values/​tests.rs Adds decimal parsing and precision regression tests.
crates/​iceberg/​src/​spec/​values/​literal.rs Uses checked extraction for string and JSON decimals.
crates/​iceberg/​src/​spec/​values/​decimal_utils.rs Applies exponents and validates precision before conversion.
crates/​iceberg/​src/​spec/​values/​datum.rs Routes decimal constructors through shared validation.

Review effort: Balanced

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decimal parsing loses positive exponents and accepts oversized values

2 participants