Skip to content

fix(ilm): reject unknown import fields and accept S3-shaped rule JSON - #367

Merged
overtrue merged 3 commits into
rustfs:mainfrom
nhattran998:fix/ilm-import-strict-dialect
Sep 9, 2026
Merged

fix(ilm): reject unknown import fields and accept S3-shaped rule JSON#367
overtrue merged 3 commits into
rustfs:mainfrom
nhattran998:fix/ilm-import-strict-dialect

Conversation

@nhattran998

Copy link
Copy Markdown
Contributor

Problem

rc ilm rule import silently drops unknown rule fields. A JSON file whose keys don't match rc's camelCase dialect imports successfully, reports ✓ Imported 1 lifecycle rule(s), and sends a degraded rule to the server — which then rejects it with a confusing error.

Real-world sequence (this is how I hit it, originally misdiagnosed as a server bug in rustfs/rustfs#7456):

$ cat lifecycle.json
{"rules":[{"id":"cleanup","status":"Enabled","filter":{"prefix":"v1/"},"abort_incomplete_multipart_upload":{"days_after_initiation":1}}]}

$ rc ilm rule import alias/bucket lifecycle.json
✗ Failed to set lifecycle rules: set_bucket_lifecycle: Network error: HTTP 400:
  InvalidArgument: Rule must have at least one of Expiration, Transition,
  NoncurrentVersionExpiration, NoncurrentVersionTransition, or DelMarkerExpiration

The server is correct — rc parsed the file, dropped filter and abort_incomplete_multipart_upload as unknown fields, and PUT an actionless rule. Anyone pasting S3-shaped JSON (a natural thing to do, since it mirrors PutBucketLifecycleConfiguration) gets this exact failure, and nothing points at the real cause.

Fix

LifecycleRuleInput (and the nested input structs) now:

  1. Accept the alternative spellings instead of silently ignoring them:
    • snake_case (abort_incomplete_multipart_upload_days) and S3/PascalCase (AbortIncompleteMultipartUploadDays) for every field;
    • the S3 JSON shape: nested filter (Prefix / Tag / And with tags and size bounds) flattened onto rc's flat filter fields, nested abortIncompleteMultipartUpload: { daysAfterInitiation }, and expiration.ExpiredObjectDeleteMarker.
  2. Fail loudly on unknown fields (deny_unknown_fields), naming the offending key — a wrong-dialect file is now a parse error, never a silent degradation.
  3. Reject ambiguous combinations: flat + nested spellings of the same value (prefix and filter.prefix, flat and nested abort days) error unless identical; a filter with more than one top-level predicate (Prefix + Tag) is rejected like the server does.

An empty filter: {} stays valid (the server documents it as "applies to every object").

Verification

  • New crates/core/tests/lifecycle_import_dialect.rs: 10 tests covering the camelCase dialect (regression guard), snake_case, PascalCase, S3-shaped filter/abort/marker acceptance, And flattening, unknown-field rejection, and both conflict cases.
  • Added a writer round-trip assertion for abort-only rules in lifecycle_xml.rs (the existing round-trip test constructed an abort value but never asserted its XML).
  • cargo fmt --all --check ✅, cargo clippy -p rc-core --all-targets -- -D warnings ✅, cargo test --workspace ✅ (all suites green).
  • End-to-end against a live rustfs/rustfs:v1.0.0-rc.5 node: the exact JSON from the problem section now imports and round-trips:
✓ Imported 1 lifecycle rule(s) to bucket 'trust-evidence'.
$ rc ilm rule export alias/trust-evidence
{"rules":[{"id":"cleanup","status":"Enabled","prefix":"v1/","abortIncompleteMultipartUploadDays":3}]}

Notes

`rc ilm rule import` silently dropped unknown rule fields, so a JSON
file using S3-shaped keys (nested filter/abortIncompleteMultipartUpload,
snake_case) imported a degraded, actionless rule that the server then
refused with a confusing "rule must have an action" error.

Import now accepts the rc camelCase dialect plus the common alternative
spellings (snake_case, S3/PascalCase, and the S3 JSON shape with a
nested Filter and AbortIncompleteMultipartUpload action), rejects
ambiguous flat+nested combinations, and fails loudly on unknown fields
instead of dropping them.
@nhattran998

Copy link
Copy Markdown
Contributor Author

cc @overtrue

@overtrue overtrue left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes because valid S3 size-only lifecycle filters are rejected during JSON import. Both standalone ObjectSizeGreaterThan and ObjectSizeLessThan cases were reproduced against this commit. The 11 existing lifecycle unit tests pass, but both regression probes fail with unknown-field errors. Please support these predicates and add regression coverage.

#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
#[serde(rename_all = "camelCase", deny_unknown_fields)]
struct LifecycleFilterInput {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Support standalone S3 object-size filter predicates. S3 permits ObjectSizeGreaterThan or ObjectSizeLessThan directly under Filter, alongside the existing Prefix, Tag, and And alternatives. For example, {"Rules":[{"ID":"size-filter","Status":"Enabled","Filter":{"ObjectSizeGreaterThan":1024},"Expiration":{"Days":30}}]} is valid S3-shaped input, but this new input type rejects it with an unknown-field error because neither size predicate is defined. ObjectSizeLessThan fails identically. This prevents importing valid size-only lifecycle rules even though the existing XML writer already supports them. Please add both fields to LifecycleFilterInput, include them in predicate counting and flattening, and cover both standalone forms with tests.

S3 permits ObjectSizeGreaterThan and ObjectSizeLessThan directly under
Filter, alongside Prefix, Tag, and And. The strict input dialect rejected
these valid size-only filters with an unknown-field error.

Add both predicates to LifecycleFilterInput (with the usual PascalCase and
snake_case aliases), count them in the Filter predicate check, and flatten
them onto rc's flat size fields. Update the multi-predicate error message
and its test, and cover both standalone forms with regression tests.
@nhattran998

Copy link
Copy Markdown
Contributor Author

Addressed in e19c6dc: LifecycleFilterInput now accepts ObjectSizeGreaterThan/ObjectSizeLessThan (plus snake_case aliases) as standalone Filter predicates, they're counted in the one-predicate check, and flatten onto rc's flat size fields. Added regression tests for both standalone forms, including your exact PascalCase example — 12/12 dialect tests, cargo clippy -p rc-core --all-targets -- -D warnings, and full rc-core suite (278+ tests) all green. Thanks for catching this — valid size-only filters were indeed being rejected.

@overtrue
overtrue merged commit 110a665 into rustfs:main Sep 9, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants