Skip to content

fix(storage): honor --enable-rapid-writes flag during object overwrites for pirlo buckets - #5004

Open
vipnydav wants to merge 4 commits into
masterfrom
vipin-honor-flag-for-writing-pirlo-objects
Open

fix(storage): honor --enable-rapid-writes flag during object overwrites for pirlo buckets#5004
vipnydav wants to merge 4 commits into
masterfrom
vipin-honor-flag-for-writing-pirlo-objects

Conversation

@vipnydav

Copy link
Copy Markdown
Member

Description

When overwriting an existing object, GCSFuse currently copies the original object's storage class into the new request. If an object with the RAPID storage class in a pirlo bucket is overwritten while the GCSFuse mount flag --enable-rapid-writes is set to false, the request fails with an InvalidArgument error. This occurs because the combination of using the Resumable upload API (which is used when rapid writes are disabled) and explicitly passing the RAPID storage class is rejected by GCS.

This PR fixes the issue by explicitly clearing the storage class (req.StorageClass = "") for pirlo buckets when rapid writes are disabled. This ensures GCSFuse strictly honors the mount flag rather than inheriting the original object's RAPID storage class, allowing the overwritten pirlo bucket object to correctly fall back to the bucket's default storage class instead.

Link to the issue in case of a bug fix.

b/543390252

Testing details

  1. Manual - NA
  2. Unit tests - NA
  3. Integration tests - Automated

Any backward incompatible change? If so, please explain.

No.

@vipnydav
vipnydav requested a review from a team as a code owner August 10, 2026 04:18
@vipnydav vipnydav added execute-integration-tests Run only integration tests execute-integration-tests-on-zb To run E2E tests on zonal bucket. labels Aug 10, 2026
@github-actions github-actions Bot added the remind-reviewers Auto remind reviewers in attention set for review post 24hrs of inactivity on PR. label Aug 10, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates CreateObject and CreateObjectChunkWriter in internal/storage/bucket_handle.go to explicitly clear req.StorageClass when rapid writes are disabled, preventing the inheritance of the RAPID storage class from overwritten objects. The reviewer suggested refactoring the if/else if conditionals to switch statements to avoid calling bh.BucketType() twice and to write more idiomatic Go code.

Comment thread internal/storage/bucket_handle.go Outdated
Comment thread internal/storage/bucket_handle.go Outdated
@codecov

codecov Bot commented Aug 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.82%. Comparing base (0673d19) to head (771edbc).
⚠️ Report is 11 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5004      +/-   ##
==========================================
+ Coverage   83.81%   83.82%   +0.01%     
==========================================
  Files         174      174              
  Lines       21376    21382       +6     
==========================================
+ Hits        17916    17924       +8     
+ Misses       2779     2778       -1     
+ Partials      681      680       -1     
Flag Coverage Δ
unittests 83.82% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

// If rapid writes are disabled, we explicitly clear the storage class to prevent
// inheriting the RAPID storage class from an overwritten object and write it to
// the bucket's default storage class.
switch bh.BucketType().Pirlo {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can extract this logic into a common method and use at both the places

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should avoid overriding variables in a different function as it is difficult to main in long run, which method is modifiying what and the order.

i would suggest adding a method getSTorageClassForCreateObject and use this in the CreateObject method
req.storageClass = getStorageClassForCreateObject(req)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Updated the helper method to return the storage class and assign it to req.StorageClass directly in the caller.

@github-actions

Copy link
Copy Markdown

Hi @vadlakondaswetha, your feedback is needed to move this pull request forward. This automated reminder was triggered because there has been no activity for over 24 hours. Please provide your input when you have a moment. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

execute-integration-tests Run only integration tests execute-integration-tests-on-zb To run E2E tests on zonal bucket. remind-reviewers Auto remind reviewers in attention set for review post 24hrs of inactivity on PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants