feat(vector): support vector values in deduplicate primary-key tables

[ChaomingZhangCN]

Purpose

Linked issue: #197

Support VECTOR as an ordinary value column in Parquet-backed deduplicate primary-key tables.

This change:

• Adds Arrow FixedSizeList support to merge-tree row conversion, memory estimation, spill, and compaction paths.
• Supports VECTOR insert, same-key update, read, projection, and null value transitions.
• Supports forced spill with updates to the same key across multiple spill runs.
• Supports adding or dropping VECTOR columns and reading files written with older schemas.
• Rejects incompatible VECTOR dimension and element-type changes.
• Rejects VECTOR as a primary key, partition key, bucket key, sequence field, sequence-group ordering field, or sorting field.
• Rejects VECTOR with lookup, partial-update, aggregation, first-row, and other unsupported merge modes.
• Keeps primary-key tables incompatible with row-tracking Data Evolution.
• Limits the initial primary-key table support to Parquet.

Tests

Added or extended unit and integration coverage for:

• Non-null and null FixedSizeList row conversion.
• Invalid VECTOR dimensions.
• Merge-tree memory estimation and spill.
• Primary-key schema validation for supported and unsupported VECTOR configurations.
• Deduplicate insert, same-key update, projection, and read.
• VECTOR-to-null and null-to-VECTOR updates.
• Same-key updates across multiple spill runs.
• Full compaction with VECTOR values.
• Add/drop VECTOR schema evolution, including field-ID reordering.
• Incompatible VECTOR dimension and element-type evolution.
• Existing row-tracking Data Evolution VECTOR behavior.

Validation completed:

• Focused FixedSizeList, schema validation, primary-key read/write, spill, compaction, schema-evolution, and Data Evolution tests passed.
• Full unittest target passed: 28/28 test suites.
• clang-format checks passed.
• git diff --check passed.

API and Format

No public API under include/ is changed.

No new storage format or protocol is introduced. VECTOR remains Parquet-only and continues using the existing Parquet LIST representation, restored as Arrow
FixedSizeList in memory.

Documentation

Updated the VECTOR support matrix and primary-key table documentation to describe:

• Deduplicate-only primary-key table support.
• VECTOR value-column restrictions.
• Unsupported key, ordering, lookup, and merge-engine configurations.
• Parquet-only scope.

Generative AI tooling

Generated-by: OpenAI Codex CLI 0.151.0 (GPT-5)