[mbutrovich]

Which issue does this PR close?

• Closes Decide whether the Arrow manifest readers return an iterator of batches #3647.
• Closes Return manifests and manifest lists as pyarrow tables from pyiceberg-core #3648.
• Part of [EPIC] Fast manifest reads for PyIceberg through pyiceberg-core #3262 (Task 3).

What changes are included in this PR?

• read_manifest_entries (feat(arrow): read manifest entries into Arrow with arrow-avro #3388) and read_manifest_list (feat(arrow): read manifest lists into Arrow with arrow-avro #3631) haven't been released, so their signatures change without a deprecation.

• The batch size is 1024 rows. An entry's size grows with the table's column count through the column stats maps. These are the Arrow sizes of 8 MB manifests written by PyIceberg with stats on every long column. The stats compress well, which fits the most entries in 8 MB.

  Table columns	Entries in an 8 MB manifest	Arrow size per entry	One batch (main)	Largest batch of 8192	Largest batch of 1024
  12	61,173	1.4 KB	85.2 MB	10.5 MB	1.3 MB
  100	9,935	7.3 KB	71.2 MB	70.0 MB	8.8 MB
  1000	810	86 KB	68.2 MB	68.2 MB (whole manifest)	68.2 MB (whole manifest)

  Wider tables have larger entries, so fewer fit in an 8 MB manifest. At 1000 columns, a manifest holds fewer than 1024 entries and reads as one batch at either size.

Are these changes tested?

• Unit tests and doctests in crates/iceberg/src/arrow/manifest.rs.
• Python tests in bindings/python/tests/test_manifest.py, which compare the binding's output with PyIceberg's own readers.

AI Disclosure

I wrote this PR with help from Claude but understand and support the core approach, implementation, and test coverage.