[Baunsgaard]

Which issue does this PR close?

• Closes Reading a name-mapped map fails with "Map field must have exactly 2 fields" when none of the map value's leaves are in the file #3557.

What changes are included in this PR?

Parquet can only read a map when both its key and value are projected. When a map value is a struct and none of its leaves match the table (a name-mapped physical name that is not in the mapping, or embedded ids that are not in the table schema), only the key was selected. Converting that pre-projected schema failed with Map field must have exactly 2 fields. Iceberg Java reads the same files.

1. Convert each selected leaf on its own instead of converting a pre-projected schema that can contain a key-only map. Duplicate field ids in the file are still rejected.
2. After the mask is built, walk the Parquet schema (detecting maps the same way parquet-rs does). If a map's key is read but none of its value leaves are, also read the value leaves. They match no table field, so the value reads as a struct of NULLs (fix(arrow): read nested struct fields as null for files without ids or name mapping #3437).
3. Only struct values are rescued. A primitive value is matched by position and would be cast to the table type, so that case keeps failing as before.
4. Leaves rejected by type_promotion_is_valid are never rescued. If every value leaf is rejected, the read fails with DataInvalid instead of a Parquet "partial projection" error.

Only reads that failed before select extra columns. A map whose value is read but whose key is not still fails as on main.

Selected leaf indices come from Arrow filter_leaves, while map key/value ranges are computed from the Parquet schema. Those two leaf numberings disagree only when a nested ListView (or similar) appears before the map in the file. That mismatch already mis-projects columns on apache/iceberg-rust main today (#3609); open PR #3624 fixes it. This PR does not change that behavior.

Related open work:

• fix(arrow): keep struct presence when its requested children are missing #3438 covers the embedded-id form of this failure, not the name-mapped case in Reading a name-mapped map fails with "Map field must have exactly 2 fields" when none of the map value's leaves are in the file #3557.
• fix(arrow): use Parquet leaf indices for projection #3624 avoids the "Map field must have exactly 2 fields" conversion error by flattening selected leaves, but without the map-value rescue Reading a name-mapped map fails with "Map field must have exactly 2 fields" when none of the map value's leaves are in the file #3557 still fails in Parquet with "partial projection of MapArray".

When the read failed

Typical case:

Table (with name mapping):          File (no field ids):
  2: m map<                        m: map<
       3: key string,                   key: string,
       4: value struct<                 value: struct<
            5: x int                        old_x: int   // not in mapping
         >>                              >>
         

Here x should result in null, not as it is now crashing.

Are these changes tested?

Yes, in arrow::reader::projection:

• test_name_mapped_map_value_with_unmapped_child_reads_null: name-mapped x reads values; old_x (Reading a name-mapped map fails with "Map field must have exactly 2 fields" when none of the map value's leaves are in the file #3557) reads a struct of NULLs.
• test_map_value_with_unmatched_embedded_id_reads_null: file id 5, table id 6. Also checks that a read projecting only id leaves the unprojected map alone.
• test_map_value_with_invalid_promotion_is_not_cast: int to string is rejected with DataInvalid; with an unmatched sibling the map reads and the rejected leaf is NULL.
• test_map_primitive_value_with_unmatched_id_is_not_cast: primitive value with a mismatched id errors instead of casting "12" to 12.

The first three fail without the fix; the last checks that a mismatched primitive value is not added to the projection.

AI Disclosure

Cursor assisted with the implementation, regression tests, and local review. I reviewed the resulting changes.

[Copilot]

🟡 Changes recommended

Critical projection issues can silently lose data through duplicate ancestor IDs and allow nested list/map casts to bypass type-promotion validation.

2 open issues

• Preserve duplicate-ID validation for selected leaves and ancestors · New
• Validate positional nested types before rescuing leaves · New

What changed in this PR

This PR addresses #3557 in the Arrow reader: projecting a map fails when none of its value struct’s leaves match the table schema. It aims to preserve map readability so missing struct fields can be returned as NULLs.

Changes:

• Converts selected leaves individually to avoid converting incomplete map schemas.
• Adds map value leaves when needed, excluding leaves rejected by type-promotion checks.
• Adds regression tests for name mapping, unmatched field IDs, and incompatible value types.

The PR description could be shortened by removing the implementation walkthrough and repeated test details.

File	Description
crates/​iceberg/​src/​arrow/​reader/​projection.rs	Updates leaf conversion and map projection handling, with regression tests.

_{Review effort: Balanced}