fix(storage): prefer compact data sets over legacy ones - #960
fix(storage): prefer compact data sets over legacy ones#960silent-cipher wants to merge 2 commits into
Conversation
commit: |
| // A fresh devnet has no legacy history, so every data set is compact. | ||
| legacyPieceStorageIdLimit: 1n, |
There was a problem hiding this comment.
For review: legacyPieceStorageIdLimit is hardcoded per-chain rather than read live from PDPVerifier.legacyPieceStorageIdLimit(). Hardcoding avoids an extra RPC round-trip on every dataset-selection call, but means a future on-chain change requires a manual SDK patch release rather than propagating automatically. Flagging for a second opinion.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
synapse-dev | 234c881 | Commit Preview URL Branch Preview URL |
Sep 11 2026, 07:00 AM |
There was a problem hiding this comment.
🟡 Changes recommended
A bigint-to-number conversion in dataset sorting can misorder datasets for large IDs, breaking deterministic preference ordering.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR updates dataset selection in both synapse-core (pure provider selection) and synapse-sdk (on-chain dataset resolution) to prioritize compact datasets over legacy ones when both match the requested metadata, using a chain-configured cutoff (legacyPieceStorageIdLimit) instead of additional RPC reads.
Changes:
- Add
legacyPieceStorageIdLimitto chain configuration and plumb it into provider selection inputs. - Update dataset ranking/selection logic to prefer compact datasets first, then apply existing piece-presence + age ordering within each tier.
- Add regression tests covering “compact beats legacy even if legacy has pieces” in both core and SDK paths.
File summaries
| File | Description |
|---|---|
| packages/synapse-sdk/src/test/storage.test.ts | Adds SDK-level test ensuring compact dataset is selected even when a legacy dataset has pieces. |
| packages/synapse-sdk/src/storage/context.ts | Splits datasets into compact/legacy tiers during provider-specific dataset resolution and prefers compact tier. |
| packages/synapse-core/test/select-providers.test.ts | Adds test asserting compact datasets are preferred when legacyPieceStorageIdLimit is provided. |
| packages/synapse-core/test/find-matching-data-sets.test.ts | Adds tests covering compact-vs-legacy ordering and intra-tier ordering rules. |
| packages/synapse-core/src/warm-storage/select-providers.ts | Passes legacyPieceStorageIdLimit into matching-dataset selection. |
| packages/synapse-core/src/warm-storage/location-types.ts | Extends provider selection input types/docs with legacyPieceStorageIdLimit. |
| packages/synapse-core/src/warm-storage/find-matching-data-sets.ts | Updates matching dataset sort order to prioritize compact datasets before legacy. |
| packages/synapse-core/src/warm-storage/fetch-provider-selection-input.ts | Sources legacyPieceStorageIdLimit from chain config via asChain(client.chain). |
| packages/synapse-core/src/devnet/index.ts | Sets devnet chain’s legacyPieceStorageIdLimit. |
| packages/synapse-core/src/chains.ts | Adds legacyPieceStorageIdLimit to the FilecoinChain interface and chain definitions. |
Review details
- Files reviewed: 10/10 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Prefer matching compact data sets during automatic and explicit provider selection, falling back to legacy data sets when no compact match exists. Read migration cutoffs from chain configuration without additional RPC calls.
closes #925