feat: support PAGE_ROW_COUNT_LIMIT for parquet format

[lucasfang]

Purpose

Linked issue: close #225

Support splitting Parquet data pages by row count, aligned with parquet-mr's parquet.page.row.count.limit (PARQUET-1414). Previously the Parquet writer split pages only by byte size (parquet.page.size), so a single page could contain a very large number of rows, leading to coarse page-index filtering granularity, read amplification, and page layouts inconsistent with files written by the Java side. This PR introduces a row count limit (default 20000, same as parquet-mr); a page is finished when either the row count or the byte size limit is reached first.

Changes:

• cmake_modules/arrow.diff: patch arrow's parquet::WriterProperties to add the data_page_row_count_limit builder method, getter, and the constant DEFAULT_DATA_PAGE_ROW_COUNT_LIMIT = 20000; extend the page-split condition in column_writer.cc with a num_buffered_rows_ >= properties_->data_page_row_count_limit() check in addition to the byte-size check.
• src/paimon/format/parquet/parquet_format_defs.h: add the table option key PARQUET_PAGE_ROW_COUNT_LIMIT = "parquet.page.row.count.limit" (the key name is identical to parquet-mr's).
• src/paimon/format/parquet/parquet_writer_builder.cpp: PrepareWriterProperties reads the option and applies it to the arrow WriterProperties, falling back to the default 20000 when unset.
• src/paimon/format/parquet/parquet_format_writer_test.cpp: update the TestMemoryControl assertions — with the default row-count page split in effect, all-null pages are RLE-encoded and finish early, so the writer peak memory stays far below the budget; for the non-null case the budget upper bound is relaxed to 2.5x (the budget is mostly held as finished page buffers, which BufferedPageWriter copies into its in-memory sink when the row group is flushed, transiently doubling the footprint; ~2.2x measured).

Note: the limit is checked at write batch granularity, so a page may exceed the limit by up to one write batch (unlike parquet-mr, which enforces the limit exactly).

Tests

• UT: ParquetWriterBuilderTest.DefaultPrepareWriterProperties adds a default-value assertion (data_page_row_count_limit == 20000); ParquetWriterBuilderTest.PrepareWriterProperties adds an assertion that the option parquet.page.row.count.limit=40000 is propagated.
• UT: ParquetFormatWriterTest.TestMemoryControl adapts the memory assertions to the new default page-split behavior.
• Actual results: paimon-parquet-format-test 187/187 passed, paimon-write-inte-test 117/117 passed, paimon-read-inte-test 278/278 passed.

API and Format

No change to the public API under include/. Adds the table option parquet.page.row.count.limit (default 20000). The storage format stays compatible: only the data page layout of newly written files changes (more pages, bounded rows per page); no new format feature is introduced and older readers can read the files normally.

Documentation

New table option parquet.page.row.count.limit: a best-effort limit on the number of rows in a single data page, aligned with parquet-mr's option of the same name (PARQUET-1414), default 20000; a page is finished when either this row count or the byte size (parquet.page.size) is reached first. Unlike parquet-mr, which enforces the limit exactly, the limit here is only checked at write-batch granularity, so a page may exceed it by up to one write batch (e.g., with the default limit 20000 and the default write batch size 1024, a page holds at most 20480 rows).

Generative AI tooling

Generated-by: Qoder

[zjw1111]

The <= 0 validation in 54dda55 looks good. One remaining point on the semantics of the new option: the row limit is best effort here, not the hard bound parquet-mr gives, and its granularity is currently tied to an unrelated option. Details inline.