perf(storage): optimize Folder.UpdateTime to int64 nanoseconds and direct proto conversion - #5001
perf(storage): optimize Folder.UpdateTime to int64 nanoseconds and direct proto conversion#5001kislaykishore wants to merge 1 commit into
Conversation
…rect proto conversion ## Summary This commit refactors `gcs.Folder.UpdateTime` from a 24-byte `time.Time` struct to an 8-byte `int64` (unix nanoseconds) and optimizes `GCSFolder()` to perform direct mathematical conversion from protobuf `controlpb.Folder` timestamp fields (`seconds * 1e9 + nanos`) without allocating intermediate `time.Time` objects. ## Struct Sizing & Go Heap Class Reduction `gcs.Folder` shrinks from 40 B to 24 B, dropping across the Go heap allocator class boundary from 48 B to 24 B—saving **24 bytes (-50.0%) of real heap memory per folder**. | Struct / Type | Baseline Size | Optimized Size | Struct Delta | Go Heap Class | Heap Allocation Delta | RAM Saved (per 1M folders) | RAM Saved (per 10M folders) | |---|:---:|:---:|:---:|:---:|:---:|:---:|:---:| | `gcs.Folder` | 40 B | **24 B** | -16 B (-40.0%) | 48 B -> **24 B** | **-24 B (-50.0%)** | **24.0 MB** | **240.0 MB** | ## Benchmark Performance Measured via `benchstat` (n=10, p < 0.05): | Benchmark Target | Master Baseline | Optimized Branch | CPU Latency Delta | Baseline Memory | Branch Memory | Allocation Delta | |---|:---:|:---:|:---:|:---:|:---:|:---:| | `GCSFolder_WithTimestamp` | 95.60 ns/op | **83.42 ns/op** | **-12.75% (p=0.000)** | 96 B/op, 2 allocs | **72 B/op, 2 allocs** | **-25.00% B/op** | | `GCSFolder_NilTimestamp` | 88.29 ns/op | **79.38 ns/op** | **-10.10% (p=0.000)** | 96 B/op, 2 allocs | **72 B/op, 2 allocs** | **-25.00% B/op** | | `Alloc_Folder` | 32.74 ns/op | **25.95 ns/op** | **-20.75% (p=0.000)** | 48 B/op, 1 alloc | **24 B/op, 1 alloc** | **-50.00% B/op** | | `Alloc_FolderSlice_1000` | 4.387 µs/op | 5.035 µs/op | ~0.0% | 40.00 KiB/op, 1 alloc | **24.00 KiB/op, 1 alloc** | **-40.00% B/op** | ## Verification - Unit tests: `go test -v ./internal/storage/gcs/...` & `go test -v ./internal/storage/fake/...` - Benchmarks: `go test -bench=BenchmarkGCSFolder -benchmem ./internal/storage/gcs/...`
There was a problem hiding this comment.
Code Review
This pull request refactors the Folder struct's UpdateTime field from time.Time to int64 (representing nanoseconds) and updates the fake storage bucket and GCS folder implementations accordingly. It also introduces mutex locking in RenameFolder and adds new tests and a benchmark. The review feedback suggests using an explicit integer constant 1000000000 instead of 1e9 for clarity in nanosecond conversions, and replacing b.Loop() in the benchmark with a traditional loop to ensure compatibility with Go versions older than 1.24.
| // Setting the parameters in Folder and doing conversions as necessary. | ||
| var updateTime int64 | ||
| if ts := attrs.GetUpdateTime(); ts != nil { | ||
| updateTime = ts.GetSeconds()*1e9 + int64(ts.GetNanos()) |
There was a problem hiding this comment.
Using 1e9 (which is an untyped float constant) in integer arithmetic can be less clear and potentially confusing. It is more idiomatic in Go to use an untyped integer constant like 1_000_000_000 or 1000000000 for nanosecond conversions.
| updateTime = ts.GetSeconds()*1e9 + int64(ts.GetNanos()) | |
| updateTime = ts.GetSeconds()*1000000000 + int64(ts.GetNanos()) |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #5001 +/- ##
==========================================
+ Coverage 83.81% 83.83% +0.01%
==========================================
Files 174 174
Lines 21376 21381 +5
==========================================
+ Hits 17916 17924 +8
+ Misses 2779 2777 -2
+ Partials 681 680 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Description
This PR optimizes
gcs.Folder.UpdateTimeby refactoring it from a 24-bytetime.Timestruct to an 8-byteint64(unix nanoseconds), and enhancesGCSFolder()to perform direct mathematical conversion from protobufcontrolpb.Foldertimestamp fields (seconds * 1e9 + nanos) without allocating intermediatetime.Timeobjects.Struct Sizing & Go Heap Class Reduction
gcs.Foldershrinks from 40 B to 24 B, dropping across the Go heap allocator class boundary from 48 B to 24 B.gcs.FolderBenchmark Performance
Measured via
benchstat(n=10, p < 0.05):GCSFolder_WithTimestampGCSFolder_NilTimestampAlloc_FolderAlloc_FolderSlice_1000Link to the issue in case of a bug fix.
N/A
Testing details
make buildand verified formatting/linter with 0 issues.go test -v ./internal/storage/gcs/...andgo test -v ./internal/storage/fake/....Any backward incompatible change? If so, please explain.
N/A