GH-51659: [R] Expose ignore_extra_columns and pad_short_rows CSV pars… - #51662
rajaryan2007 wants to merge 1 commit into
Conversation
|
|
|
|
…V parse options Arrow C++ 26.0.0 adds pad_short_rows and ignore_extra_columns to arrow::csv::ParseOptions. Neither could be set from R, so add them to csv_parse_options() and pass them through from csv.cpp. CsvParseOptions$create is an alias of csv_parse_options(), so both entry points accept the new arguments.
b52067b to
23a8d17
Compare
|
|
4 similar comments
|
|
|
|
|
|
|
|
|
|
2 similar comments
|
|
|
|
1 participant
Rationale for this change
Arrow C++ 26.0.0 added two new
arrow::csv::ParseOptionsfields:ignore_extra_columns(GH-50967: [C++] Allow CSV reader to ignore extra columns in rows with more columns #51118): rows with more columns than expected havethe extra columns dropped instead of erroring
pad_short_rows(GH-50925: [C++] Allow CSV reader to pad rows with missing trailing fields #50926): rows with fewer columns than expected are paddedwith nulls instead of erroring
Neither could be set from R, so
read_csv_arrow()andopen_csv_dataset()error on ragged CSV with no way to opt into the C++ behaviour.
Are these changes tested?
Yes. test-csv.R gains pad_short_rows and ignore_extra_columns parse options, covering each option alone, both together, the plain-list form, and the CsvParseOptions$create() entry point. It also asserts ragged input still errors by default, pinning the new defaults.
Verified locally with R 4.4.3: the full testthat suite passes (6,561 assertions, 0 failures, 0 errors), and the new test was confirmed to fail against the pre-change r/src/csv.cpp.
Are there any user-facing changes?
Additive: csv_parse_options() and CsvParseOptions$create() accept two new arguments, both defaulting to FALSE to match the C++ defaults, so existing code is unaffected.
Was AI used for this PR?
i used ai for build and some someother help .
PR code and description written by:
Reviewed before submission by: