fix(caching): only cache negative entries from authoritative listings - #5025
fix(caching): only cache negative entries from authoritative listings#5025nk412 wants to merge 1 commit into
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
There was a problem hiding this comment.
Code Review
This pull request modifies the caching behavior in fast_stat_bucket.go to prevent windowed or incomplete listings (such as those with a StartOffset or ContinuationToken) from triggering negative caching. This ensures that incomplete listings do not overwrite valid positive cache entries and cause incorrect ENOENT errors. Unit and integration tests have been added to verify this behavior. There are no review comments, so no additional feedback is provided.
An empty ListObjects result proves a directory does not exist only when it covers the complete prefix. A windowed listing (StartOffset set) or a continuation page covers part of the namespace, and inserting a negative entry from it can overwrite a valid positive implicit-directory entry, serving false ENOENT for the whole subtree until the negative TTL expires. Pass the originating ListObjectsRequest to insertListing and skip the negative entry when the request has a StartOffset or continuation token, or when the response carries a continuation token. Fixes GoogleCloudPlatform#5018
0c3256d to
16eda58
Compare
Description
Prevent empty windowed or paginated object listings from creating a negative stat-cache entry for an entire implicit-directory prefix.
fastStatBucket.insertListingpreviously treated every empty listing as evidence that the requested directory did not exist. That is not authoritative when the request usesStartOffsetor a continuation token, or when the response contains a continuation token. In those cases, the listing represents only part of the namespace, and inserting a negative entry can overwrite a valid positive directory entry and cause transientENOENTresponses.This change:
ListObjectsRequesttoinsertListing;Link to the issue in case of a bug fix.
Fixes #5018
Testing details
gcs.NotFoundError: negative cache entry for dir/and passes with this change.make buildalso passes.golang:1.26.5Docker:go test ./internal/storage/caching -count=1CGO_ENABLED=0 go test ./internal/storage/... -count=1go test -race ./internal/storage/caching -count=1go vet ./internal/storage/cachingCGO_ENABLED=0 go build ./...TestIntegration_WindowedListingDoesNotPoisonExistingDirectory, which exercisesfastStatBucketwith the real LRU/stat cache and fake GCS backend. A live FUSE/GCS end-to-end test was not run.Any backward incompatible change? If so, please explain.
N/A. The changed method is internal, and no exported API, configuration, or existing authoritative-listing behavior changes.