fix(sql): align S3 Select behavior with RustFS - #370
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
Background and user impact
The
rc sqlcommand diverged from the current RustFS S3 Select implementation in three places: CSV input record delimiters were not configurable, Parquet scan ranges were rejected locally even though RustFS supports them, and scan ranges on compressed input reached the server before failing. Select service errors could also collapse to the unhelpfulservice errortext.Root cause
S3 Select validation was duplicated between the CLI and AWS SDK adapter, the core CSV input model did not expose
RecordDelimiter, and service-error classification preferred the generic SDK wrapper over modeled error metadata.Solution
--csv-input-record-delimiteroption with RustFS-compatible one- or two-byte validation.rc-coreand validate before alias lookup or network access.ScanRangerequests.ScanRangerequests for GZIP/BZIP2 input locally while preserving RustFS's accepted{ start: 0, end: none }no-op form.Busy/SlowDownas retryable network failures.Validation
cargo fmt --all --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspace