diff --git a/docs/source/io_pattern_design.md b/docs/source/io_pattern_design.md
new file mode 100644
index 0000000000..88eac9d220
--- /dev/null
+++ b/docs/source/io_pattern_design.md
@@ -0,0 +1,1246 @@
+---
+orphan: true
+---
+## 5.4.1 设计概述
+
+```
+flowchart TD
+ subgraph 推理框架层 ["推理框架层 (vLLM/SGLang)"]
+ MC["MooncakeConnector
(vLLM v1)"]
+ HC["HiCache Connector
(SGLang)"]
+ PC["Prefix Cache Manager"]
+ end
+
+ subgraph CFM ["CFM (Cache Flow Manager)"]
+ COL["Collector
(数据采集)"]
+ ANA["Analyzer
(模式分析)"]
+ ENG["Policy Engine
(策略决策)"]
+ COL --> ANA --> ENG
+ EV["Eviction Ops"]
+ PF["Prefetch Ops"]
+ AD["Admission Ops"]
+ ENG --> EV & PF & AD
+ end
+
+ MC -->|"CFM Client
(采集/策略/预取)"| COL
+ HC -->|"CFM Client"| COL
+ PC -->|"CFM Client"| COL
+
+ subgraph CVM ["CVM (Cache View Manager)"]
+ VIEW["视图计算 / 发布 / 系统事件感知 / 全局 KV 映射表"]
+ end
+
+ EV --> VIEW
+ PF --> VIEW
+ AD --> VIEW
+
+ subgraph 存储层 ["存储层"]
+ L0["L0: HBM
(UB2PCIe/d2h)"]
+ L1["L1: Host DRAM/SSD
(计算节点本地内存/SSD(xds)"]
+ L2["L2: Segment DRAM
(池化内存 URMA mem)"]
+ L3["L3: Nof SSD
(SSU/远端池化 SSD)"]
+ end
+
+ VIEW --> L0
+ VIEW --> L2
+ VIEW --> L3
+ L0 <-.->|"tier down/up"| L1
+ L1 <-.->|"tier down/up"| L2
+ L2 <-.->|"offload/promotion"| L3
+```
+
+**模块总体架构图**
+
+```
+classDiagram
+ class IoPatternCollector {
+ <>
+ +ReportInferenceMetrics(metrics) void
+ +RecordAccess(key, record) void
+ +RecordStorageMetric(metric) void
+ +GetSnapshot() IoPatternSnapshot
+ }
+
+ class IoPatternAnalyzer {
+ <>
+ +AnalyzePattern(snapshot) PatternResult
+ +DetectWorkloadType(window) WorkloadType
+ +CalculateConfidence(key) float
+ }
+
+ class PolicyEngine {
+ <>
+ +ExecutePolicy(context, tier, bytes, trace, admissions) PolicyResult
+ }
+
+ class EvictionOps {
+ <>
+ +Evaluate(context, tier, bytes) EvictionPlan
+ }
+
+ class PrefetchOps {
+ <>
+ +Evaluate(context, trace) PrefetchPlan
+ }
+
+ class AdmissionOps {
+ <>
+ +Evaluate(object, tier, context) AdmissionResult
+ }
+
+ class CfmClient {
+ +ReportInferenceMetrics(metrics) void
+ +ReceivePolicy指令() void
+ +ExecutePrefetch(candidates) void
+ }
+
+ class ScoreBasedEviction {
+ +Evaluate(context, tier, bytes) EvictionPlan
+ }
+
+ class TraceBasedPrefetch {
+ +Evaluate(context, trace) PrefetchPlan
+ }
+
+ class PrefixMatchAdmission {
+ +Evaluate(object, tier, context) AdmissionResult
+ }
+
+ PolicyEngine *-- EvictionOps : contains
+ PolicyEngine *-- PrefetchOps : contains
+ PolicyEngine *-- AdmissionOps : contains
+ IoPatternCollector --> IoPatternAnalyzer : reports
+ IoPatternAnalyzer --> PolicyEngine : analyzes
+ CfmClient --> IoPatternCollector : reports metrics
+ CfmClient --> PolicyEngine : receives policy
+ EvictionOps <|.. ScoreBasedEviction : implements
+ PrefetchOps <|.. TraceBasedPrefetch : implements
+ AdmissionOps <|.. PrefixMatchAdmission : implements
+```
+
+## 5.4.2 IO Pattern 三层架构
+
+IO Pattern 模块采用**采集层 -> 分析层 -> 策略层**的三层架构。
+
+### 5.4.2.1 采集层 (IO Pattern Collector)
+
+采集层负责从各数据源采集原始 IO 指标,采用异步上报机制避免阻塞数据路径。
+
+**采集来源分三层:**
+
+| 采集层 | 数据源 | 采集指标 | 现有代码锚点 |
+| ----------------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------- |
+| 推理框架层 | vLLM MooncakeConnector / SGLang HiCache Connector | prefix match length, request priority, token 序列, recompute cost | `mooncake_connector_v1.py`, SGLang hicache connector |
+| SuperCache SDK 层 | Client / Master / SubMaster | Get/Put/Tier 命中率, 访问时序, key 频率, 前缀树深度/fanout, 副本分布, 迁移 ETA, 写路径指标 (batch_size, overwrite_ratio) | `client_service.cpp`, `master_service.cpp`, `local_hot_cache.cpp` |
+| 存储后端层 | SSD/Nof Segment | 读写带宽, 读写延迟, GC 状态, 盘内 Superblock 布局, 容量水位 | `client_metric.h:SsdMetric`, `allocation_strategy.h:SsdMetricsProvider` |
+
+**采集机制设计:**
+
+1. **轻量级埋点**:复用和扩展现有 `CountMinSketch`(频率统计)、`SsdMetric`(SSD 延迟/吞吐)、`storage_backend.h:last_access_ns_`(最后访问时间)等埋点,避免重复建设
+2. **异步上报**:CFM Client 定期异步上报指标至 SubMaster,采用 batch 聚合减少 RPC 开销。上报间隔自适应负载(低负载 100ms,高负载退避至 500ms-1s)
+3. **全局聚合**:SubMaster 聚合各节点上报的指标,维护全局 token 指标流动视图
+4. **采样降级**:在高负载场景下支持采样率动态调整,优先保障数据路径性能
+5. **多租户隔离**:指标按 `TenantId` 分桶采集,避免高频租户淹没低频租户,沿用现有 `CountMinSketch` 的 `tenant_id.MakeScopedKey` 模式
+
+固定 100ms 上报间隔在大规模集群下可能产生可观开销。采用自适应间隔:
+
+| 负载状态 | 上报间隔 | 触发条件 |
+| -------- | -------- | ----------------------------------------- |
+| 低负载 | 100ms | mem_used_ratio < 50% |
+| 中负载 | 200ms | 50% <= mem_used_ratio < 80% |
+| 高负载 | 500ms | mem_used_ratio >= 80% |
+| 极高负载 | 1000ms | mem_used_ratio >= 95% 或 RPC 延迟 > 100ms |
+
+>
+> 高负载时拉长间隔减少 RPC 开销,但保持最低 1s 上报频率确保策略时效性。量化估算:4000 节点集群,100ms 间隔下每秒 40000 RPC,单 RPC ~2KB,总带宽 ~80MB/s
+
+#### 5.4.2.1.1 指标采集与上报流程
+
+```
+sequenceDiagram
+ participant INF as 推理框架 (vLLM/SGLang)
+ participant CFM as CFM Client
+ participant SUB as SubMaster
+
+ INF->>CFM: 1. 请求完成/前缀匹配
+ CFM->>SUB: 2. 批量上报指标 (InferenceMetrics)
+ CFM->>SUB: 3. SDK 层埋点 (AccessRecord)
+ CFM->>SUB: 4. 存储后端指标 (StorageMetric)
+ Note over SUB: 5. 全局聚合
IoPatternSnapshot
+```
+
+#### 5.4.2.1.2 指标分类
+
+IO Pattern 采集指标分为六大类,对应分级缓存淘汰/准入/预取流程的采集指标定义:
+
+**时序指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| --------------------- | --------- | ------------------------ | ---------- | ----------------------------------- |
+| `last_access_time` | timestamp | 最近一次访问时间 | SDK 层 | `storage_backend.h:last_access_ns_` |
+| `access_count_window` | uint32 | 最近时间窗口内访问次数 | SDK 层 | `count_min_sketch.h:CountMinSketch` |
+| `idle_time` | duration | = now - last_access_time | 分析层计算 | - |
+
+**价值指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| ---------------- | ------ | ------------------------------ | ---------- | ---------------- |
+| `recompute_cost` | float | 重新计算时间 (token 数 / 时间) | 推理框架层 | connector 层估算 |
+| `block_size` | uint64 | 数据大小 (bytes) | SDK 层 | object metadata |
+| `token_count` | uint32 | token 数 | 推理框架层 | connector 层 |
+
+**结构指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| -------------------------- | ------ | ---------------------------- | ---------- | -------------------- |
+| `prefix_depth` | uint32 | 前缀深度 (prefix tree level) | 推理框架层 | prefix cache manager |
+| `prefix_fanout` | uint32 | 共享该前缀的请求/分支数量 | 推理框架层 | prefix cache manager |
+| `match_length` | uint32 | 前缀匹配长度 | 推理框架层 | connector 层 |
+| `continuous_prefix_length` | uint32 | 连续前缀长度 | 推理框架层 | connector 层 |
+
+**副本指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| --------------------- | -------- | -------------------------- | ---------- | -------------------------- |
+| `replica_tiers` | bitmap | 当前在哪些层有副本 (L0-L3) | SDK 层 | `master_service.h:Replica` |
+| `transfer_eta` | duration | 迁移路径预计耗时 | 分析层计算 | - |
+| `ssd_replica_exists` | bool | SSD 层是否有副本 | SDK 层 | replica metadata |
+| `other_replica_count` | uint32 | 其他层副本数 | SDK 层 | replica metadata |
+
+**状态指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| -------- | ---- | ------------ | -------- | ----------------------------- |
+| `active` | bool | 是否正在使用 | SDK 层 | `local_hot_cache.h:ref_count` |
+| `pinned` | bool | 是否不可迁移 | SDK 层 | promotion task pinning |
+
+**存储后端指标**
+
+| 指标名 | 类型 | 描述 | 采集来源 | 现有代码锚点 |
+| ------------------- | --------- | ----------------- | ---------- | ---------------------------------------------------------------------------- |
+| `ssd_read_latency` | histogram | SSD 读延迟分布 | 存储后端层 | `client_metric.h:SsdMetric` |
+| `ssd_write_latency` | histogram | SSD 写延迟分布 | 存储后端层 | `client_metric.h:SsdMetric` |
+| `ssd_gc_status` | enum | SSD GC 状态 | 存储后端层 | Nof TGT |
+| `mem_used_ratio` | float | DRAM 内存水位比例 | SDK 层 | `MasterMetricManager::get_global_mem_used_ratio()` (Master 侧全局 DRAM 水位) |
+| `ssd_used_bytes` | int64 | SSD 已用容量 | 存储后端层 | `allocation_strategy.h:SsdMetricsProvider` |
+
+#### 5.4.2.1.3 指标采集接口
+
+```
+// IO Pattern Collector 接口 (新增, 位于 include/io_pattern_collector.h)
+class IoPatternCollector {
+ public:
+ virtual ~IoPatternCollector() = default;
+
+ // 推理框架层指标 (通过 CFM Client 上报)
+ virtual void ReportInferenceMetrics(const InferenceMetrics& metrics) = 0;
+
+ // SDK 层指标 (内部埋点)
+ virtual void RecordAccess(const std::string& key,
+ const AccessRecord& record) = 0;
+
+ // 存储后端层指标
+ virtual void RecordStorageMetric(const StorageMetric& metric) = 0;
+
+ // 获取聚合后的指标快照
+ virtual IoPatternSnapshot GetSnapshot() const = 0;
+};
+
+struct AccessRecord {
+ std::string key;
+ std::chrono::steady_clock::time_point access_time;
+ uint64_t block_size;
+ ReplicaType replica_type;
+ bool is_hit;
+ std::chrono::microseconds latency;
+};
+
+struct InferenceMetrics {
+ std::string session_id;
+ uint32_t prefix_depth;
+ uint32_t prefix_fanout;
+ uint32_t match_length;
+ uint32_t continuous_prefix_length;
+ uint32_t token_count;
+ float recompute_cost;
+ uint8_t request_priority;
+};
+```
+
+### 5.4.2.2 分析层 (IO Pattern Analyzer)
+
+分析层对采集的原始指标进行模式识别和特征提取,输出结构化的 IO Pattern 描述。
+
+**分析能力:**
+
+| 分析类型 | 描述 | 输入指标 | 输出 | 对应需求 |
+| ------------- | ------------------------------ | ------------------------------------------------------ | ------------------------------------ | -------------- |
+| 热度分析 | 基于 LFU/滑动窗口的频率统计 | access_count_window, last_access_time | hot/cold 分类, 频率评分 | 淘汰/准入/预取 |
+| 前缀分析 | Prefix tree 深度和 fanout 分析 | prefix_depth, prefix_fanout, match_length | 前缀共享度, 预取候选 | 预取/准入 |
+| 时序预测 | 访问间隔和空闲时间分析 | idle_time, access pattern, access_count_window | 空闲评分, 预取优先级 | 淘汰/Tier down |
+| 代价评估 | 重计算代价和迁移代价评估 | recompute_cost, token 数, block_size, transfer_eta | 代价评分, 迁移 ROI | 淘汰/Tier up |
+| 访问模式识别 | 顺序/随机、大包/小包、读写比 | IO size distribution, access sequence | 模式分类 (SEQ/RANDOM/KV_LOOKUP等) | 缓存分区/分流 |
+| 副本分析 | 多层副本分布分析 | replica_tiers, active/pinned | 副本冗余度, 迁移安全性 | 淘汰/Tier down |
+| 写路径分析 | 写入模式分析 | write_batch_size, write_burst, overwrite_ratio | 写穿风险, GC 预警 | 准入/GC |
+| Workload 识别 | 推理场景类型自动识别 | token_count, prefix_fanout, block_size, frequency 分布 | workload_type (Code Agent/推荐/对话) | 策略模板选择 |
+
+**Workload Type 感知策略模板**
+
+不同推理场景的 IO Pattern 差异巨大,单一通用评分公式无法覆盖所有场景。
+
+**配置参数:**
+
+| 参数 | 默认值 | 说明 |
+| -------------------------------------- | ------------- | ------------------------------------------------------------------ |
+| `workload_detection_window_sec` | 60 | workload 识别滑动窗口大小 |
+| `workload_detection_method` | `auto` | 识别方法:`auto`(阈值+聚类)、`threshold`(仅阈值)、`kmeans`(仅聚类) |
+| `workload_template_transition_windows` | 3 | 模板切换过渡窗口数,控制平滑度 |
+| `workload_mixed_load_mode` | `per_session` | 混合负载处理:`per_session`(按会话标记)、`global`(全局统一) |
+
+**运维观测:**
+
+| 指标 | 描述 |
+| --------------------------------------- | ------------------------ |
+| `workload_current_type` | 当前识别的 workload type |
+| `workload_type_switch_count` | workload type 切换次数 |
+| `workload_detection_latency_us` | 单次识别延迟 |
+| `workload_template_transition_progress` | 模板过渡进度 (0.0-1.0) |
+
+**Workload 识别机制:**
+
+不同推理场景的 KVCache 访问模式差异巨大,单一通用评分公式无法覆盖所有场景。IO Pattern 分析层基于滑动窗口内的指标分布统计,自动识别 workload type 并切换对应策略模板。
+
+**Workload Type 特征矩阵:**
+
+| Workload Type | 典型场景 | 访问特征 | 识别信号 |
+| ------------- | ------------------------------ | ------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
+| Code Agent | Cursor/Copilot 长程代码生成 | 长上下文(>32K token)、高前缀复用、大 block_size(>512KB)、多轮访问为主 | 高 token_count + 低 prefix_fanout + 大 block_size + 低 frequency |
+| 生成式推荐 | 京东/字节 GR 精排召回 | 高频小 block(<128KB)、高复用、密集访问、低重计算代价 | 低 token_count + 高 frequency + 小 block_size + 低 recompute_cost |
+| 多轮对话 | ChatGPT 类对话、Agent 工具调用 | 中等 block、高前缀共享、渐进式增长、prefix cache 命中率高 | 中 token_count + 高 prefix_fanout + 高 match_length + 高 recompute_cost |
+
+**识别算法:**
+
+分析层维护一个滑动窗口(默认 60s),统计窗口内所有请求的 `token_count`/`prefix_fanout`/`block_size`/`frequency` 四维分布。采用两阶段识别:
+
+```
+阶段 1: 特征提取
+ for each request in window:
+ feature_vector = (median(token_count), p90(prefix_fanout),
+ median(block_size), median(frequency))
+
+阶段 2: 分类决策
+ if feature_vector matches阈值规则:
+ -> 直接分类 (快速路径, 延迟 < 1ms)
+ else:
+ -> K-means 聚类 (慢速路径, 延迟 < 10ms, 用于混合负载场景)
+```
+
+**阈值规则(快速路径):**
+
+| 判定条件 | → Workload Type |
+| ---------------------------------------------------------------------------------- | ----------------------- |
+| `median(token_count) > 16KB && p90(prefix_fanout) > 16 && p90(match_length) > 256` | Code Agent |
+| `median(block_size) < 128KB && median(frequency) > 20` | 生成式推荐 |
+| `p90(prefix_fanout) > 16 && p90(match_length) > 256` | 多轮对话 |
+| 不满足以上任一 | 混合负载 → K-means 聚类 |
+
+**策略模板对照:**
+
+每种 workload type 对应一组完整的策略参数模板,覆盖淘汰/准入/预取/Tier 四个维度:
+
+| 策略维度 | Code Agent | 生成式推荐 | 多轮对话 |
+| ------------- | ----------------------------------------- | --------------------------------------- | -------------------------------------- |
+| **预取** | 保守(仅 prefix > 512 预取,best_effort) | 激进(prefix > 64 预取,wait_complete) | 前缀优先(prefix > 256 预取,timeout) |
+| **淘汰** | 激进(低 idle_thres,快速释放 L0) | 保守(高 idle_thres,保留热数据) | 前缀感知(prefix_fanout 高权重保留) |
+| **准入** | 低阈值(access_count > 2 即准入) | 高阈值(access_count > 20 才准入) | 前缀准入(match_length > 128 即准入) |
+| **Tier down** | 快速降级(L0→L2 跳级,跳过 L1) | 缓慢降级(L0→L1→L2 逐层) | 前缀亲和(共享前缀的 block 同层迁移) |
+| **淘汰权重** | α=0.8, γ=0.2, δ=0.3, ε=0.1 | α=0.3, γ=0.8, δ=0.2, ε=0.1 | α=0.5, γ=0.4, δ=0.6, ε=0.8 |
+
+**模板切换机制:**
+
+```
+flowchart TD
+ WIN["滑动窗口指标统计
(60s)"]
+ FEAT["特征提取
4 维分布向量"]
+ RULE["阈值规则匹配"]
+ KMEANS["K-means 聚类
(混合负载)"]
+ CLASSIFY["Workload Type 判定"]
+ TEMPLATE["策略模板加载
(淘汰/准入/预取/Tier 参数)"]
+ APPLY["应用至 Policy Engine"]
+
+ WIN --> FEAT --> RULE
+ RULE -->|"匹配成功"| CLASSIFY
+ RULE -->|"不匹配"| KMEANS --> CLASSIFY
+ CLASSIFY --> TEMPLATE --> APPLY
+```
+
+**切换平滑性:** workload type 变化时,策略参数不是瞬间切换,而是通过加权过渡(新旧模板权重在 3 个窗口周期内从 100:0 渐变到 0:100),避免策略突变导致缓存抖动。
+
+**混合负载处理:** 当 K-means 识别出多种 workload type 共存时(如同一集群同时服务对话和推荐),采用 per-session workload 标记——在请求入口处根据 session 特征打标签,各 session 独立使用对应模板,而非全局统一。
+
+### 5.4.2.3 策略层 (IO Pattern Policy Engine)
+
+策略层基于分析层的输出,通过可注册的 Ops 接口驱动各缓存机制:
+
+```
+flowchart TD
+ PE["Policy Engine"]
+
+ subgraph EvictionOps ["EvictionOps (淘汰策略)"]
+ LRU["LRU"]
+ LFU["LFU"]
+ SBE["ScoreBased
(L0-L3 四层)"]
+ end
+
+ subgraph PrefetchOps ["PrefetchOps (预取策略)"]
+ BE["BestEffort"]
+ TO["Timeout"]
+ WC["WaitComplete"]
+ TB["TraceBased"]
+ end
+
+ subgraph AdmissionOps ["AdmissionOps (准入策略)"]
+ FREQ["Frequency"]
+ PM["PrefixMatch"]
+ WM["Watermark"]
+ CA["CostAware"]
+ end
+
+ PE --> EvictionOps
+ PE --> PrefetchOps
+ PE --> AdmissionOps
+```
+
+## 5.4.3 缓存机制集成与关键流程
+
+IO Pattern 模块不是重写现有机制,而是在现有机制之上增加统一的数据采集和分析层,通过 Ops 抽象接口驱动各机制。
+
+### 5.4.3.2 淘汰 (Eviction)
+
+**现有机制**:`EvictionStrategy` 抽象类(`eviction_strategy.h`)提供 LRU 和 FIFO 两种实现;`storage_backend.h` 中基于 `last_access_ns_` 维护 LRU 索引。
+
+**IO Pattern 增强**:引入基于评分的淘汰策略 (ScoreBasedEviction),按4层缓存层级分别使用不同评分公式。所有指标先经归一化处理(`normalize(x) = x / max_observed_x`,映射到 `[0, 1]`),消除量纲差异后再加权求和。归一化基准基于滑动窗口(默认 60s)内的最大观测值动态更新。
+
+```
+flowchart TD
+ SNAP["IO Pattern Snapshot"]
+ NORM["归一化处理
norm(x) = x / max_observed"]
+ PE["Policy Engine
选择层级策略"]
+ SNAP --> NORM --> PE
+
+ PE --> L0S["L0 HBM Evict"]
+ PE --> L1S["L1 Host DRAM/SSD Evict"]
+ PE --> L2S["L2 Segment DRAM Evict"]
+ PE --> L3S["L3 Nof SSD Evict"]
+
+ L0S --> L0F["α\*norm(idle) - γ\*norm(freq)
- δ\*norm(recompute) - ε\*norm(fanout)"]
+ L1S --> L1F["α\*norm(idle) - γ\*norm(freq)
+ δ\*norm(lower_replica)
- ε\*norm(fanout) - ζ*norm(recompute)"]
+ L2S --> L2F["α\*norm(idle) - γ\*norm(freq)
+ δ\*norm(lower_replica)
- ε\*norm(fanout) - ζ*norm(recompute)"]
+ L3S --> L3F["α\*norm(idle)\*norm(block_size)
- γ\*norm(freq) - δ\*norm(recompute)
+ η*norm(other_replica)"]
+
+ L0F --> L0OUT["-> L1"]
+ L1F --> L1OUT["-> L2"]
+ L2F --> L2OUT["-> L3"]
+ L3F --> L3OUT["-> 丢弃"]
+```
+
+**各层淘汰策略说明:**
+
+| 层级 | 评分侧重 | 淘汰去向 | 说明 |
+| ---------------- | --------------------------------------------------------------------- | -------- | ------------------------- |
+| L0 HBM | `idle_time` 主导,`recompute_cost`/`prefix_fanout` 高权重保留 | → L1 | HBM 最贵,冷数据快速降级 |
+| L1 Host DRAM/SSD | 下层已有副本可安全淘汰,`prefix_fanout`/`recompute_cost` 高的数据保留 | → L2 | 本地 DRAM/SSD 到远端 DRAM |
+| L2 Segment DRAM | `prefix_fanout`/`recompute_cost` 高的数据保留 | → L3 | 池化内存到远端 SSD |
+| L3 Nof SSD | `block_size` 大 + 其他层已有副本优先淘汰 | → 丢弃 | 最底层,无下降空间 |
+
+**集成方式**:扩展现有 `EvictionStrategy` 接口,新增 `ScoreBasedEvictionStrategy`,由 Policy Engine 根据层级动态选择策略。
+
+### 5.4.3.3 准入 (Admission)
+
+**现有机制**:
+
+- Client 侧:`CountMinSketch` + `admission_threshold_` 频率准入(`client_service.cpp`),仅频繁访问的 key 提升 hot cache
+- Master 侧:Promotion-on-Hit 的 `promotion_admission_threshold_` 频率门控 + watermark 门控(`master_service.cpp`)
+
+**IO Pattern 增强**:扩展准入策略为多层逐级准入控制,每层提升需满足对应条件:
+
+| 准入路径 | 条件 | 说明 |
+| -------- | -------------------------------------------------------------- | ----------------------------------------------------- |
+| L3→L2 | `access_count_window >= threshold` | 频率达标才从 Nof SSD 提升至 Segment DRAM |
+| L2→L1 | `access_count_window >= threshold && upper_space <= max_space` | 频率达标且上层有空间才提升 Segment DRAM→Host DRAM/SSD |
+| L1→L0 | `max_length >= threshold (64)` | 前缀长度达标才从 Host DRAM/SSD 提升至 HBM |
+
+>
+> SSD→HBM 跨层直达(跳过中间层)仅由推理框架 prefix cache 命中时触发,Mooncake 侧不自主执行跨层晋升到 HBM。
+
+```
+flowchart TD
+ REQ["访问请求"]
+ ANA["IO Pattern Analyzer
计算准入条件"]
+ REQ --> ANA
+
+ ANA --> P1["L3→L2
access_cnt >= thres"]
+ ANA --> P2["L2→L1
access_cnt >= thres
&& upper_space <= max"]
+ ANA --> P3["L1→L0
max_length >= 64"]
+
+ P1 --> FA["频率准入
(CountMin Sketch)"]
+ P2 --> FA2["频率+空间准入
(Frequency + Watermark)"]
+ P3 --> PA["前缀准入
(PrefixMatch Admission)"]
+```
+
+**集成方式**:扩展现有 `CountMinSketch` 准入逻辑,新增 `PrefixMatchAdmission` 和 `CostAwareAdmission` 策略。
+
+### 5.4.3.4 Tier Down / SSD Offload
+
+**现有机制**:
+
+- `enable_ssd_offload` + `ssd_offload_path` 配置 SSD offload 路径(`real_client.cpp`)
+- `offload_on_evict` 模式:在淘汰时延迟 offload 到 LOCAL_DISK(`master_service.cpp`)
+- `offload_force_evict`:超过 offload cap 时直接淘汰不 offload
+
+**IO Pattern 增强**:基于热度阈值的逐级 tier down,数据按 L0→L1→L2→L3 顺序逐层降级:
+
+```
+flowchart TD
+ START["L0 HBM 容量/水位检测"]
+ C1{"idle_time >= L0 cold_thres || frequency < L0 hot_thres ?"}
+ C2{"L2 seg_dram_avail ?"}
+ C3{"L3 nof SSD avail ?"}
+ C4{"L1 host DRAM<= thres ?"}
+ C5{"xds available ?"}
+ C6{"idle_time >= L1 cold_thres || frequency < L1 hot_thres ?"}
+ C7{"idle_time >= L2 cold_thres || frequency < L2 hot_thres ?"}
+ TD_L1A["L1 Host DRAM"]
+ TD_L1B["L1 SSD(xds)"]
+ TD_L2["L2 Segment DRAM"]
+ TD_L3["L3 Nof SSD"]
+
+ START --> C1
+ C1 -->|是| C4
+ C4 -->|是| TD_L1A
+ C4 -->|否| C5
+ C5 -->|是| TD_L1B
+ C5 -->|否| C2
+ C2 -->|是| TD_L2
+ C2 -->|否| C3
+ C3 -->|否| WAIT["下层均不可用
等待重试 / 强制 evict"]
+ C3 -->|是| TD_L3
+ TD_L1A --> C6
+ C6 -->|是| C2
+ TD_L2 --> C7
+ C7 -->|是| C3
+```
+
+>
+> 当所有下层均不可用时,数据暂留当前层并等待下层恢复,或触发强制 evict 释放空间。冷数据不会保留在高速层——高速层是最昂贵的资源,冷数据必须逐级降级。
+
+### 5.4.3.5 Tier Up / Promotion-on-Hit
+
+**现有机制**:
+
+- `promotion_on_hit` 模式(`master_service.cpp:379`):Get 观察到 LOCAL_DISK-only key 时队列异步拷贝回 MEMORY
+- `CountMinSketch` 频率门控(`master_service.cpp:6944`)
+- watermark 门控:DRAM 低于 `eviction_high_watermark_ratio_` 才允许 promotion
+- `promotion_queue_limit` + `promotion_max_per_heartbeat` 控制 promotion 速率
+- `PromotionCandidate` 跟踪 + 重试 + TTL 过期
+
+**IO Pattern 增强**:IO Pattern 分析层为 promotion 提供更丰富的决策输入:
+
+- 前缀匹配度:高前缀匹配的 key 优先 promotion
+- 重计算代价:高 recompute_cost 的 key 优先 promotion
+- 迁移 ETA:根据带宽和 block_size 估算 transfer_eta,避免迁移耗时过长
+
+>
+> Mooncake 侧 promotion 仅执行逐级提升(L3→L2→L1),不自主晋升到 L0 HBM。L0 HBM 层的数据加载由推理框架 prefix cache 命中时自主触发。
+
+```
+flowchart TD
+ START["prefix cache 命中/预取触发/get"]
+ CALC["Tier Up Priority 计算:
priority = w0 * recompute_cost
+ w1 * continuous_prefix
+ w2 * request_priority
- w3 * transfer_eta"]
+ SORT["按优先级排序"]
+ EXEC["执行逐级 tier up
L3→L2→L1"]
+
+ START --> CALC --> SORT --> EXEC
+```
+
+### 5.4.3.6 预取 (Prefetch)
+
+**现有机制**:当前无显式预取机制,Promotion-on-Hit 在 Get 命中 LOCAL_DISK 时异步提升到 MEMORY。
+
+**IO Pattern 增强**:新增 `PrefetchOps` 抽象,SubMaster 根据 trace 和置信阈值生成预取器:
+
+**预取触发条件:** 低速层 prefix match length > 阈值 (256) 时触发预取至上一层(如 L3→L2、L2→L1)
+
+**预取策略(三种模式):**
+
+| 策略 | 描述 | 适用场景 |
+| --------------- | ----------------------------------- | -------------------- |
+| `best_effort` | check & match,无论是否完成立即返回 | 对 TTFT 时延敏感业务 |
+| `timeout` | 预取完成或超时立即返回 | 兼顾时延和命中率 |
+| `wait_complete` | 死等数据加载完成 | 追求极致命中率 |
+
+分析层基于 trace 历史命中率和置信阈值生成策略输入。置信度 = 滑动窗口内命中次数 / 总访问次数,低于阈值时不触发操作避免误判。例如预取器生成:
+
+- SubMaster 根据 trace 历史 + 置信阈值(如 `confidence > 0.6 && prefix match length > 256`)生成预取器
+- 置信度低于阈值时不触发操作,避免误判导致的缓存污染
+- 置信阈值精确定义和各策略默认值详见
+
+**置信度计算:** 基于滑动窗口内的历史命中率,衡量当前预测的可信程度。
+
+```
+confidence = hit_count_in_window / total_access_in_window
+```
+
+**置信阈值应用:**
+
+| 策略 | 置信阈值 | 含义 | 默认值 |
+| -------- | ----------------------------------------------- | --------------------------------------- | ------------------------------------ |
+| 预取触发 | `confidence > 0.6 && match_length > 256` | 历史命中率 > 60% 且前缀匹配足够长才预取 | prefix_threshold=256, confidence=0.6 |
+| 准入提升 | `confidence > 0.5 && access_count >= threshold` | 历史命中率 > 50% 且频率达标才提升 | confidence=0.5 |
+| 淘汰保守 | `confidence > 0.8` 时降低淘汰权重 | 高置信热数据更保守淘汰 | confidence=0.8 |
+
+>
+> 置信度低于阈值时不触发操作,避免误判导致的缓存污染。置信度窗口默认 60s,可通过 `confidence_window_sec` 配置。
+
+预取流程仅看 `match_length > 256` 触发
+
+```
+flowchart TD
+ START["低速层 prefix match
(L1-L3)"]
+ C1{"match_length > 256 ?"}
+ NOP["不预取"]
+ SEL["选择预取策略"]
+
+ START --> C1
+ C1 -->|否| NOP
+ C1 -->|是| SEL
+ SEL --> BE["best_effort"]
+ SEL --> TO["timeout"]
+ SEL --> WC["wait_complete"]
+```
+
+- `max_prefetch_ratio`:预取占用带宽上限比例,默认 20%,可通过 `prefetch_max_bw_ratio` 配置
+- 带宽不足时延迟重试,而非直接丢弃预取请求
+
+**集成方式**:在 SubMaster 中新增预取器,根据 IO Pattern 分析层的置信阈值异步预取 key 至上层。
+
+## 5.4.5 上层推理框架对接
+
+### 5.4.5.1 vLLM 集成
+
+**现有对接**:`MooncakeConnector`(`mooncake_connector_v1.py`)实现 vLLM `KVConnectorBase_V1` 接口,支持 PD disaggregation(Prefill/Decode 分离)。
+
+>
+> **上层框架改动**:vLLM 侧无需改动。`MooncakeConnector` 作为 vLLM 的 out-of-tree connector(通过 `--kv_connector_module_path` 加载),在 connector 内部新增 CFM Client 调用即可上报指标和接收策略指令,不涉及 vLLM scheduler/engine 接口变更。vLLM v0.13.0+ 已内置 mooncake connector,后续可考虑将 CFM Client 合入上游。
+
+**IO Pattern 对接增强**:
+
+```
+flowchart TD
+ VLLM["vLLM Engine"]
+
+ subgraph MC ["MooncakeConnector (KVConnectorBase_V1)"]
+ GNMT["get_num_new_matched_tokens()
上报 match_length"]
+ USA["update_state_after_alloc()
上报 prefix_depth"]
+ RF["request_finished()
上报 token_count, recompute_cost"]
+ end
+
+ subgraph CFMC ["CFM Client (新增)"]
+ RIM["IoPatternCollector
.ReportInferenceMetrics()"]
+ POP["PrefetchOps
接收预取指令"]
+ AOP["AdmissionOps
接收准入策略"]
+ end
+
+ VLLM --> MC
+ VLLM --> CFMC
+ GNMT --> RIM
+ USA --> RIM
+ RF --> RIM
+```
+
+**采集对接**:在 `MooncakeConnector` 中增加 CFM Client 调用,将以下指标上报至 IO Pattern Collector:
+
+- `match_length`:prefix cache 命中长度(来自 `get_num_new_matched_tokens()`)
+- `prefix_depth` / `prefix_fanout`:前缀树结构(来自 `update_state_after_alloc()` 及 prefix cache manager)
+- `token_count`:请求 token 数(来自 `request_finished()`)
+- `recompute_cost`:重计算代价估算(connector 侧基于 token_count 和模型 FLOPS 估算)
+- `request_priority`:请求优先级(connector 层从 request metadata 提取)
+
+**策略对接**:CFM Client 接收 Policy Engine 的策略指令:
+
+- 预取指令:根据 prefix match length > 256 触发异步预取
+- 准入指令:根据频率/前缀匹配控制数据提升层级
+- 淘汰指令:根据淘汰评分驱动 L0-L3 层间淘汰
+
+### 5.4.5.2 SGLang 集成
+
+**现有对接**:SGLang HiCache 通过 `--hicache-storage-backend: mooncake` 将 Mooncake 作为存储后端,支持 layer_first / page_first 布局。
+
+**IO Pattern 对接增强**:
+
+```
+flowchart TD
+ SGL["SGLang Engine"]
+
+ subgraph HCC ["HiCache Connector"]
+ HR["hicache-ratio
容量配比"]
+ HML["hicache-mem-layout
layer_first / page_first"]
+ HIO["hicache-io-backend
direct / async"]
+ end
+
+ subgraph CFMS ["CFM Client (新增)"]
+ RIM2["IoPatternCollector
.ReportInferenceMetrics()"]
+ DLA["数据布局适配
(layer_first / page_first)"]
+ PAS["预取/准入/淘汰策略"]
+ end
+
+ SGL --> HCC
+ SGL --> CFMS
+ HR --> RIM2
+ HML --> DLA
+ HIO --> RIM2
+```
+
+**数据布局适配**:IO Pattern 需感知推理框架的 KV cache 布局模式,vLLM 和 SGLang 均需适配:
+
+| 框架 | 布局模式 | 描述 | IO Pattern 适配 |
+| ------ | ------------------------------ | ------------------------------------------------------------------------- | ----------------------------------------------- |
+| SGLang | `layer_first` | (2, layer, slot, num_head, head_dim) | 前缀分析按 layer 维度,预取按 layer 批量 |
+| SGLang | `page_first` | (2, page_num, layer, num_head, head_dim) | 前缀分析按 page 维度,预取按 page 批量 |
+| SGLang | `page_first_direct` | 混合模型 (Full Attention + SWA/Mamba) | 分区准入,full KV 固定分区 + SWA/mamba 灵活分配 |
+| vLLM | `page-based` (block_size 粒度) | vLLM v1 默认 page-based 布局,connector 通过 `get_kv_cache_layout()` 检测 | 前缀分析按 block 维度,预取按 block 批量 |
+| vLLM | `HMA multi-group` | 混合模型 (attention + Mamba2),`SupportsHMA` 多 group 布局 | 分组准入,各 group 独立淘汰/预取策略 |
+
+>
+> vLLM connector 已在初始化时调用 `get_kv_cache_layout()` 检测布局(`mooncake_connector_v1.py:511`),并通过 `SupportsHMA` 支持 hybrid 模型多 group 布局。SGLang 通过 `--hicache-mem-layout` 参数显式配置布局。两者均需在 CFM Client 上报时附带布局信息,供 IO Pattern 分析层选择对应的预取/准入粒度。
+
+### 5.4.5.3 CFM Client 设计
+
+CFM Client 部署在推理节点侧,作为推理框架与 SuperCache 之间的策略桥梁:
+
+```
+flowchart TD
+ subgraph CFMClient ["CFM Client"]
+ MR["Metrics Reporter
(采集上报)"]
+ PR["Policy Receiver
(策略接收)"]
+ PE["Prefetch Executor
(预取执行)"]
+ RPC["CFM RPC Channel
(to SubMaster / PrefixCache Master)"]
+ MR --> RPC
+ PR --> RPC
+ PE --> RPC
+ end
+```
+
+**职责:**
+
+1. **Metrics Reporter**:定期(100ms)批量上报推理框架指标至 SubMaster
+2. **Policy Receiver**:接收 Policy Engine 的淘汰/预取/准入策略指令
+3. **Prefetch Executor**:执行异步预取,支持 best_effort / timeout / wait_complete 三种模式
+
+## 5.4.6 Ops 抽象接口设计
+
+> **接口修订(2026-09)**:本节原始的 string/vector 简化签名仅用于
+> 查询示例,不能承载租户、字节预算、评分、Tier、超时和置信度等执行
+> 元数据。实际实现统一采用文末“Revised Ops contract”中的完整计划接口。
+
+### 5.4.6.1 EvictionOps
+
+```
+// include/eviction_ops.h (扩展现有 eviction_strategy.h)
+class EvictionOps {
+ public:
+ virtual ~EvictionOps() = default;
+
+ // 基于 IO Pattern 评分选择淘汰 key
+ virtual std::vector SelectEvictionCandidates(
+ const IoPatternSnapshot& snapshot,
+ CacheTier tier,
+ size_t target_bytes) = 0;
+
+ // 注册淘汰算法
+ static void Register(const std::string& name,
+ std::function()> factory);
+};
+
+// 已有实现: LRU, FIFO (eviction_strategy.h)
+// 新增实现: ScoreBasedEviction (L0-L3 四层不同评分公式)
+```
+
+### 5.4.6.2 PrefetchOps
+
+```
+// include/prefetch_ops.h (新增)
+class PrefetchOps {
+ public:
+ virtual ~PrefetchOps() = default;
+
+ // 基于 trace 和置信阈值生成预取候选
+ virtual std::vector GeneratePrefetchPlan(
+ const IoPatternSnapshot& snapshot,
+ const TraceHistory& trace) = 0;
+
+ // 执行预取
+ virtual ErrorCode ExecutePrefetch(
+ const std::vector& candidates,
+ PrefetchStrategy strategy) = 0;
+
+ static void Register(const std::string& name,
+ std::function()> factory);
+};
+
+enum class PrefetchStrategy {
+ kBestEffort, // 无论是否完成立即返回
+ kTimeout, // 预取完成或超时立即返回
+ kWaitComplete, // 死等数据加载完成
+};
+```
+
+### 5.4.6.3 AdmissionOps
+
+```
+// include/admission_ops.h (新增, 扩展现有 CountMinSketch 准入)
+class AdmissionOps {
+ public:
+ virtual ~AdmissionOps() = default;
+
+ // 准入决策:是否允许数据进入目标层
+ virtual AdmissionDecision CheckAdmission(
+ const std::string& key,
+ CacheTier target_tier,
+ const IoPatternSnapshot& snapshot) = 0;
+
+ static void Register(const std::string& name,
+ std::function()> factory);
+};
+
+enum class AdmissionDecision {
+ kAdmit, // 允许进入
+ kRejectFrequency, // 频率不足
+ kRejectWatermark, // 水位过高
+ kRejectPrefix, // 前缀匹配不足
+ kDefer, // 延迟决策 (记录候选)
+};
+
+// 已有实现: FrequencyAdmission (CountMinSketch)
+// 新增实现: PrefixMatchAdmission, CostAwareAdmission
+```
+
+### 5.4.6.4 Ops 注册机制
+
+```
+classDiagram
+ class EvictionOps {
+ <>
+ +SelectEvictionCandidates(snapshot, tier, bytes) vector~string~
+ +Register(name, factory) void
+ }
+ class PrefetchOps {
+ <>
+ +GeneratePrefetchPlan(snapshot, trace) vector~PrefetchCandidate~
+ +ExecutePrefetch(candidates, strategy) ErrorCode
+ +Register(name, factory) void
+ }
+ class AdmissionOps {
+ <>
+ +CheckAdmission(key, tier, snapshot) AdmissionDecision
+ +Register(name, factory) void
+ }
+ class PolicyEngine {
+ -ops_registry_ : map
+ +SelectOps(type, name) Ops
+ +ExecutePolicy(snapshot) PolicyResult
+ }
+
+ PolicyEngine --> EvictionOps : 查找/执行
+ PolicyEngine --> PrefetchOps : 查找/执行
+ PolicyEngine --> AdmissionOps : 查找/执行
+```
+
+## 5.4.7 写路径 IO Pattern
+
+### 5.4.7.1 写路径采集指标
+
+| 指标名 | 类型 | 描述 | 采集来源 |
+| ------------------- | ------ | ------------------------------ | ------------------- |
+| `write_batch_size` | uint32 | 批量写入 key 数 | SDK 层 (`BatchPut`) |
+| `write_object_size` | uint64 | 单次写入数据大小 | SDK 层 |
+| `write_burst` | bool | 是否突发写入(短时间大量 Put) | 分析层计算 |
+| `write_frequency` | uint32 | key 写入频率 | SDK 层 |
+| `overwrite_ratio` | float | 覆盖写比例 (同 key 重复 Put) | 分析层计算 |
+
+### 5.4.7.2 写路径 Pattern 对策略的影响
+
+| Pattern | 影响策略 | 处理方式 |
+| ---------- | ---------------------- | ------------------------------------------------------------------ |
+| 突发写入 | 准入:避免写穿 SSD | 突发写入期间提高 `admission_threshold`,冷数据暂留 DRAM 不 offload |
+| 高覆盖写 | 准入:跳过 SSD offload | 覆盖写比例高的 key 不 offload 到 SSD,避免无效写入 |
+| 大批量写入 | GC:提前触发 | 预估写入量,提前通知 SSD 后端准备 GC 空间 |
+| 低频写入 | 淘汰:降低保留优先级 | 低频写入的 key 在淘汰评分中 `frequency` 低,优先淘汰 |
+
+## 5.4.8 健壮性与可观测性
+
+### 5.4.8.1 失败降级
+
+IO Pattern 模块自身故障时,必须不影响数据路径,降级到现有基础机制:
+
+```
+flowchart TD
+ START["策略执行请求"]
+ C1{"IO Pattern 模块可用?"}
+ C2{"分析层响应
超时?"}
+ NORMAL["正常路径:
ScoreBasedEviction / PrefixMatchAdmission / TraceBasedPrefetch"]
+ DEGRADE["降级路径:
LRU / FIFO / FrequencyAdmission
(现有基础机制)"]
+
+ START --> C1
+ C1 -->|是| C2
+ C1 -->|否| DEGRADE
+ C2 -->|否| NORMAL
+ C2 -->|是| DEGRADE
+```
+
+| 故障场景 | 降级行为 | 触发条件 |
+| -------------- | --------------------------------- | ------------------- |
+| SubMaster 崩溃 | 回退到 Client 本地 LRU/FIFO | RPC 连续失败 > 3 次 |
+| 分析层超时 | 使用上一次成功快照 | 响应延迟 > 500ms |
+| 分析层 OOM | 丢弃 per-key 指标,仅保留全局指标 | 内存占用 > 阈值 |
+| RPC 网络抖动 | 延长上报间隔,本地缓存策略 | 丢包率 > 5% |
+
+### 5.4.8.2 反馈闭环
+
+策略执行后需评估效果并自适应调优参数,形成闭环:
+
+```
+flowchart LR
+ EXEC["策略执行
(eviction/prefetch/admission)"]
+ EVAL["效果评估
(命中率/eviction抖动/TTFT)"]
+ TUNE["参数调优
(权重/阈值自适应)"]
+ EXEC --> EVAL --> TUNE --> EXEC
+```
+
+**效果评估指标:**
+
+| 指标 | 描述 | 评估窗口 |
+| ------------------- | ------------------------------ | ------------- |
+| `hit_rate_delta` | 策略执行后命中率变化 | 60s 滑动窗口 |
+| `eviction_churn` | 淘汰抖动(刚淘汰又被访问) | 120s 滑动窗口 |
+| `ttft_delta` | TTFT 时延变化 | 30s 滑动窗口 |
+| `prefetch_accuracy` | 预取命中率(预取后是否被访问) | 60s 滑动窗口 |
+
+**参数自适应:** 当 `hit_rate_delta < 0` 持续超过 3 个评估窗口时,自动回退权重调整(如降低 `α` 权重),或切换到更保守的策略(如 ScoreBased → LRU)。
+
+### 5.4.8.3 IO Pattern 自观测
+
+IO Pattern 模块自身的运行指标,用于运维和调优:
+
+| 指标 | 描述 |
+| -------------------------------- | ----------------------------------------- |
+| `io_pattern_collect_latency_us` | 单次采集延迟 |
+| `io_pattern_analyze_latency_us` | 单次分析延迟 |
+| `io_pattern_policy_decision_qps` | 策略决策 QPS |
+| `io_pattern_strategy_hit_rate` | 策略命中率(策略命中 vs 总决策) |
+| `io_pattern_false_positive_rate` | 误判率(预取未被访问 / 淘汰后被重新加载) |
+| `io_pattern_degrade_count` | 降级次数 |
+| `io_pattern_report_drop_count` | 上报丢弃数(采样降级) |
+
+
+# IO Pattern implementation design
+
+This page records the implementation state of the IO Pattern design and is
+updated together with the code. It is intentionally separate from the original
+proposal so that unresolved decisions are visible.
+
+## Current architecture
+
+```text
+Store/Get/Put -> Collector -> bounded Analyzer -> PolicyEngine -> Ops
+ | | |-> Eviction handler
+ | | |-> Prefetch handler
+ | | `-> Admission handler
+ | `-> per-session K-means fallback
+ `-> Reporter -> authenticated CFM channel/pool
+```
+
+`MasterService` owns the runtime because it owns the authoritative replica
+metadata. Its handlers use the existing safe quota-eviction and
+promotion-on-hit queues; HBM stays inference-runtime-owned and is never moved
+by the Store master.
+
+### Current implementation architecture
+
+The following diagram is the implementation-level view. It distinguishes the
+local Store data path from the optional *remote* central CFM deployment: metric
+reporting is asynchronous, while a received CFM command is executed by the
+same storage handlers as a locally planned command. The two `MasterService`
+boxes are deployment roles, not two mandatory Mooncake service types. A normal
+deployment has one active Master (plus an optional HA standby); a separate
+central CFM Master is needed only when metrics and policy are centralized
+across multiple Masters. The roles may also be co-located for a single-Master
+deployment.
+
+```mermaid
+flowchart TB
+ subgraph producers["Metric producers"]
+ direction LR
+ inference["vLLM / SGLang bridge\nInferenceMetrics"]
+ access["Store Get/Put paths\nAccessRecord"]
+ storage["Storage and watermark paths\nStorageMetric"]
+ end
+
+ subgraph local["Reporting / policy-consuming MasterService"]
+ direction TB
+ runtime["IoPatternRuntime"]
+ collector["IoPatternCollectorImpl\nper-tenant/object aggregation\nrolling snapshot"]
+ reporter["IoPatternReporter\nbounded MetricBatch queue\nadaptive 100/200/500/1000 ms flush"]
+ analyzer["ResilientAnalyzer\nSlidingWindowAnalyzer\nbudget + timeout fallback"]
+ policy["DegradingPolicyEngine\nWorkloadPolicyEngine\nper-session templates"]
+ executor["TierOperationExecutor"]
+ feedback["PolicyFeedbackWindow +\nAdaptivePolicyTuner"]
+ admission_worker["Admission worker\nbounded deferred queue"]
+
+ runtime --> collector
+ collector --> reporter
+ collector --> analyzer
+ analyzer --> policy
+ policy --> executor
+ executor --> feedback
+ feedback -. "tune eviction weights" .-> policy
+ policy --> admission_worker
+ end
+
+ subgraph local_ops["Store-owned safe execution handlers"]
+ direction LR
+ evict["Eviction\ntenant-qualified quota eviction"]
+ prefetch["Prefetch\nLOCAL_DISK → MEMORY promotion queue"]
+ admit["Admission\npost-write retention / promotion"]
+ end
+
+ subgraph transport["Optional authenticated CFM transport"]
+ direction LR
+ codec["CfmBinaryCodec\nversioned CFM2 wire format"]
+ channel["CfmRpcChannel\nauthenticate + encode/decode"]
+ resilient["ResilientCfmChannel\nbounded retry + degradation state"]
+ rpc["CoroRpcCfmTransport\nexisting coro_rpc client pool"]
+ codec --> channel --> resilient
+ channel --> rpc
+ end
+
+ subgraph central["Central CFM MasterService"]
+ direction TB
+ rpc_service["CfmRpcService\nAuthenticate / Send / Receive /\nAcknowledge / EnqueuePolicy"]
+ service["CfmService\nauthentication + per-node bounded queues"]
+ ingress["CfmIngress\ndecode and normalize remote metrics"]
+ central_runtime["IoPatternRuntime\nCollector → Analyzer → PolicyEngine"]
+ producer_worker["PolicyProducerWorker\nproduce high-watermark eviction\nand trace-derived prefetch commands"]
+ policy_queue["Policy queue per stable node_id\ndelivery_id + ACK state"]
+ rpc_service --> service --> ingress --> central_runtime --> producer_worker --> policy_queue
+ end
+
+ inference --> runtime
+ access --> runtime
+ storage --> runtime
+ executor --> evict
+ executor --> prefetch
+ executor --> admit
+
+ reporter -->|"report_metric_batch"| channel
+ rpc -->|"authenticated RPC"| rpc_service
+ policy_queue -->|"poll_policy"| rpc
+ resilient --> client["CfmClientImpl\nPollAndDispatchPolicy"]
+ client -->|"PolicyCommand"| runtime
+ client -->|"ACK success / failure"| resilient
+
+ external_producer["External policy producer\nproducer credential"] -->|"enqueue_policy"| rpc_service
+ observability["IoPatternObservability\nlatency, hit rate, false positives,\ndegradation, report drops"] -.-> runtime
+```
+
+`CfmClientImpl` is deliberately not the normal metric-reporting entry point in
+the production wiring. `IoPatternReporter` sends metric batches directly
+through `CfmRpcChannel`; the client object owns the polling, dispatch and ACK
+loop for CFM-issued policy commands.
+
+## Implemented
+
+- `IoPatternCollectorImpl` aggregates inference, access and storage metrics by
+ tenant/object and returns deterministic snapshots.
+- `ThresholdAnalyzer` classifies Code Agent, recommendation, conversation and
+ mixed workloads and calculates continuous confidence scores.
+- `ScoreBasedEvictionOps`, `PrefixMatchAdmissionOps` and
+ `TraceBasedPrefetchOps` provide the first production policy implementations.
+- `WorkloadPolicyEngine` selects workload templates, applies real weighted
+ transition over three detection windows, and selects independent templates
+ for K-means-labelled sessions.
+- `OpsRegistry` and `RegistryPolicyEngine` resolve named policy implementations.
+- `PolicyEngine::ExecutePolicy` returns one `PolicyResult` containing eviction,
+ prefetch and admission outcomes.
+- `IoPatternReporter` provides bounded, non-blocking batches with explicit
+ report/drop counters and a transport-agnostic sink.
+- `MetricBatchTransport` defines the transport seam, and the reporter exposes
+ load-sensitive 100/200/500/1000 ms flush recommendations.
+- `IoPatternRuntime` wires collection, bounded analysis, policy execution,
+ feedback tuning and storage handlers; `MasterService` feeds it from actual
+ Get/Put/watermark paths.
+- `CfmClientImpl` dispatches received policy commands through
+ `IoPatternRuntime::ExecuteCommand`, so CFM-issued plans take the same safe
+ Store execution route as locally planned ones.
+- `CfmIngress` is the CFM-to-Store endpoint: it decodes authenticated snapshot
+ and metric-batch payloads into the runtime, and executes remote prefetch
+ plans through the same handlers.
+- `ResilientCfmChannel` adds bounded retries and consecutive-failure
+ degradation state around a concrete transport.
+- `PolicyFeedbackWindow` aggregates bounded execution-effect windows, and
+ `AdaptivePolicyTuner` adjusts eviction weights after repeated negative
+ hit-rate deltas.
+- `IoPatternObservability` provides thread-safe counters for collection and
+ analysis latency, policy hit rate, false positives, degradation and report
+ drops.
+- Its windowed snapshot also exposes strategy hit rate, false-positive rate and
+ policy decision QPS.
+- `SlidingWindowAnalyzer` keeps timestamp-bounded snapshots and computes
+ median/p90 workload features before threshold classification.
+- `IoPatternCollectorImpl` enforces an optional per-tenant key quota and
+ exposes dropped-observation counts for overload protection.
+- The vLLM connector accumulates match, allocation and completion metrics per
+ request and reports a complete layout-aware record through its optional
+ `io_pattern_bridge`. `SglangHiCacheIoPatternBridge` provides the matching
+ bounded, layout-aware adapter for HiCache request-finished/prefix hooks.
+- `TierOperationExecutor` bridges `PolicyResult` to storage-owned eviction,
+ prefetch and admission handlers and marks missing handlers as degraded.
+- `ResilientAnalyzer` caches the last successful result and falls back to it
+ (or conservative mixed mode) when analysis throws, with failure tracking.
+- `CfmBinaryCodec` defines the versioned `CFM2` protocol and fully round-trips
+ snapshots, metric batches and every policy command. `InProcessCfmRpcTransport`
+ provides authenticated embedded operation, while `CfmChannelPool` reuses and
+ fails over a bounded set of injected network channels.
+- `CfmRpcChannel::SendMetricBatch` and `MakeCfmMetricBatchSink` connect the
+ bounded Reporter to the RPC path; producers only enqueue and Flush performs
+ the transport call outside the data-path critical section.
+- `IoPatternReporter::Start/Stop` provides a background flush worker with
+ adaptive intervals; `Stop` performs a final synchronous drain.
+- `IoPatternCollectorImpl` derives write-path fields for PUT records:
+ frequency, batch size, object size, overwrite ratio and burst flag.
+- `DegradingPolicyEngine` switches to a caller-provided fallback engine after
+ repeated failures and supports explicit recovery.
+- `AdaptivePolicyTuner` also reacts to eviction churn, TTFT regression and
+ prefetch accuracy, exposes conservative mode and supports persistence
+ callbacks for tuned weights. The runtime accepts feedback samples and
+ applies the resulting weights to both global and per-session engines.
+- Analyzer execution has a single in-flight worker, timeout fallback to the
+ last safe result, and an explicit key-count budget; collector key quotas and
+ reporter bounds provide the associated overload/OOM protection.
+- Access and write frequencies use timestamped buckets pruned against a true
+ rolling 60-second cutoff. Sliding analysis deduplicates objects across
+ snapshots and enforces a hard total retained-key budget (including a single
+ oversized snapshot), so repeated high-watermark evaluations cannot multiply
+ complete snapshots without bound. CFM ingress rebases process-local monotonic
+ timestamps to receiver time, and each object also has a hard bucket-count cap.
+- Store-side eviction executes only the tenant-qualified objects selected by
+ the policy. The legacy `BatchEvict` path runs only when policy execution
+ fails, avoiding a second unplanned eviction pass.
+- Non-memory PUT completions enqueue bounded, asynchronous L1 retention/
+ promotion evaluation. This is a post-write cache-admission hook, not initial
+ replica placement: the existing `PutStart` contract selects and allocates
+ replicas before write metrics such as batch and overwrite are known.
+- CFM polling distinguishes a command, a healthy empty queue and a transport
+ error. Only transport errors contribute to consecutive-failure degradation.
+- Reporter intervals follow the documented memory/RPC load thresholds
+ (100/200/500/1000 ms), and in-process transport callbacks execute outside the
+ transport mutex.
+
+## Interface decision: complete plans versus document shorthand
+
+The proposal's shorthand methods returned only keys, candidates or a decision.
+The implementation also needs tenant identity, byte sizes, scores, confidence,
+timeout and strategy metadata. Therefore the complete plan interfaces are the
+only Ops execution seam:
+
+- `EvictionOps::Evaluate` returns `EvictionPlan`.
+- `PrefetchOps::Evaluate` returns `PrefetchPlan`.
+- `AdmissionOps::Evaluate` returns `AdmissionResult`.
+
+The former shorthand methods (`SelectEvictionCandidates`,
+`GeneratePrefetchPlan`, `ExecutePrefetch`, and `CheckAdmission`) have been
+removed from the C++ interfaces. Callers must use complete plans and
+`TierOperationExecutor` for execution.
+
+### Revised Ops contract (2026-09)
+
+The following contract supersedes the shorthand signatures in section 5.4.6:
+
+```cpp
+class EvictionOps {
+ public:
+ virtual EvictionPlan Evaluate(const PolicyContext&, CacheTier,
+ uint64_t target_bytes) const = 0;
+};
+
+class PrefetchOps {
+ public:
+ virtual PrefetchPlan Evaluate(const PolicyContext&,
+ const TraceHistory&) const = 0;
+};
+
+class AdmissionOps {
+ public:
+ virtual AdmissionResult Evaluate(const ObjectRef&, CacheTier,
+ const PolicyContext&) const = 0;
+};
+```
+
+`EvictionPlan` carries tenant-qualified objects, byte budgets and scores;
+`PrefetchPlan` carries source/target tiers, strategy, timeout and confidence;
+`AdmissionResult` carries tenant identity, target tier, decision and
+confidence. These fields are required by execution, observability and
+multi-tenant isolation and must not be collapsed into strings.
+
+`PolicyEngine::ExecutePolicy` is the single orchestration entry point and
+returns `PolicyResult` with explicit `degraded` propagation.
+
+Registry ownership is external and thread-safe. Factories return independent
+Ops instances; callers own the returned smart pointers. Concrete storage and
+RPC resources are injected through execution handlers and CFM channels.
+
+## Production CFM wiring
+
+Master registers authenticated CFM handlers on its existing `coro_rpc` port.
+`CoroRpcCfmTransport` is the production client: metric batches are delivered to
+`CfmIngress`, while policy commands use a bounded per-node queue and are polled
+by stable `node_id`. Received commands execute through
+`IoPatternRuntime::ExecuteCommand`, preserving the same storage-safe handlers as
+local policy decisions. Every report RPC also carries that `node_id`; ingress
+uses it as the authoritative storage-metric source so central aggregation does
+not merge watermarks from different Masters.
+
+Configure a central CFM receiver with `io_pattern_cfm_auth_token`. Configure each
+reporting/policy-consuming Master with:
+
+- `io_pattern_cfm_endpoint=host:port`
+- `io_pattern_cfm_node_id=` (defaults to the local CVM
+ SubMaster RPC endpoint, `rpc_address:rpc_port`)
+- the same `io_pattern_cfm_auth_token`
+- on the central receiver only, a distinct
+ `io_pattern_cfm_producer_auth_token` for policy producers
+- optional `io_pattern_cfm_timeout_ms` and
+ `io_pattern_cfm_policy_queue_capacity`
+
+An outbound Master authenticates during construction and fails startup if the
+configured CFM endpoint cannot be reached or rejects the token. At runtime the
+Reporter sends metric batches over the channel and a resilient poll loop
+dispatches queued policies. On the receiver, each accepted metric batch is put
+onto a bounded policy-production queue; the central runtime runs
+Collector -> Analyzer -> PolicyEngine asynchronously and automatically queues
+high-watermark eviction and trace-derived prefetch commands for the reporting
+`node_id`. An external policy producer may also call the registered
+`CfmRpcService::EnqueuePolicy` RPC with a target node id and an encoded
+`PolicyCommand`.
+
+Node credentials cannot use that explicit enqueue RPC; an external producer
+must authenticate with the separately configured producer credential. The
+server validates commands before enqueueing them, assigns a delivery id, and
+retains each command until the target node acknowledges successful execution.
+Its poll response distinguishes an authenticated empty queue from a rejected
+or invalid request, so authorization failures enter the normal degradation
+path. The configured capacity is enforced for both pending production work and
+policy delivery, with policy delivery bounded both per node and globally.
+
+The SGLang adapter remains framework-neutral because SGLang source is not
+vendored in this repository.
diff --git a/docs/yh/deployment.md b/docs/yh/deployment.md
new file mode 100644
index 0000000000..df0a0879a2
--- /dev/null
+++ b/docs/yh/deployment.md
@@ -0,0 +1,777 @@
+
+
+
+
+# POC项目Mooncake部署文档
+
+## 1 ETCD环境搭建
+### 1.1 安装
+
+```bash
+wget --no-check-certificate https://github.com/etcd-io/etcd/releases/download/v3.5.9/etcd-v3.5.9-linux-arm64.tar.gz
+tar -xzvf etcd-v3.5.9-linux-arm64.tar.gz
+rm etcd-v3.5.9-linux-arm64.tar.gz
+mv etcd-v3.5.9-linux-arm64/ etcd
+cd etcd/
+cp etcd etcdctl /usr/local/bin/
+```
+
+### 1.2 配置
+
+#### 1.2.1 方法1,一同创建(推荐)
+
+##### 1.2.1.1 设置system脚本
+
+```bash
+vim /etc/systemd/system/etcd.service
+```
+
+`ETCD_NAME, ETCD_INITIAL_ADVERTISE_PEER_URLS, ETCD_ADVERTISE_CLIENT_URLS`根据本机实际配置
+`ETCD_INITIAL_CLUSTER`配置集群全部节点的地址
+
+```
+[Unit]
+Description=etcd key-value store
+After=network.target
+
+[Service]
+Type=notify
+Environment=ETCD_NAME=
+Environment=ETCD_DATA_DIR=/var/lib/etcd
+Environment=ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
+Environment=ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
+Environment=ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
+Environment=ETCD_ADVERTISE_CLIENT_URLS=http://:2379
+Environment=ETCD_INITIAL_CLUSTER="=http://:2380,=http://:2380"
+Environment=ETCD_INITIAL_CLUSTER_STATE="new"
+Environment=ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
+ExecStart=/usr/local/bin/etcd
+Restart=on-failure
+RestartSec=5
+User=root
+
+[Install]
+WantedBy=multi-user.target
+```
+
+##### 1.2.1.2 清除缓存,应用配置,启动etcd服务
+
+为所有节点配置完成后,启动全部服务
+
+```bash
+systemctl disable etcd
+systemctl stop etcd
+rm -rf /var/lib/etcd
+systemctl daemon-reload
+systemctl start etcd
+systemctl enable etcd
+```
+
+##### 1.2.1.3 查看集群
+
+```bash
+etcdctl --endpoints=http://:2379 member list --write-out=table
+```
+
+#### 1.2.2 方法2,逐个创建
+
+##### 1.2.2.1 设置system脚本
+
+```bash
+vim /etc/systemd/system/etcd.service
+```
+
+主机 (ip换为本机ip)
+
+```
+[Unit]
+Description=etcd key-value store
+After=network.target
+
+[Service]
+Type=notify
+Environment=ETCD_NAME=
+Environment=ETCD_DATA_DIR=/var/lib/etcd
+Environment=ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
+Environment=ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
+Environment=ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
+Environment=ETCD_ADVERTISE_CLIENT_URLS=http://:2379
+Environment=ETCD_INITIAL_CLUSTER="=http://:2380"
+Environment=ETCD_INITIAL_CLUSTER_STATE="new"
+Environment=ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
+ExecStart=/usr/local/bin/etcd
+Restart=on-failure
+RestartSec=5
+User=root
+
+[Install]
+WantedBy=multi-user.target
+```
+
+##### 1.2.2.2 清除缓存,应用配置,启动etcd服务
+
+```bash
+systemctl stop etcd
+rm -rf /var/lib/etcd
+systemctl daemon-reload
+systemctl start etcd
+```
+
+##### 1.2.2.3 添加节点
+
+```bash
+etcdctl --endpoints=http://:2379 member add --peer-urls=http://:2380
+```
+
+有如下打印则为添加成功
+
+```
+[root@slot1 system]# etcdctl --endpoints=http://:2379 member add --peer-urls=http://:2380
+Member 6a5201d77502faa1 added to cluster 592c5fccdb3ab88c
+
+ETCD_NAME=""
+ETCD_INITIAL_CLUSTER="=http://:2380,=http://:2380"
+ETCD_INITIAL_ADVERTISE_PEER_URLS="http://:2380"
+ETCD_INITIAL_CLUSTER_STATE="existing"
+```
+
+##### 1.2.2.4 将打印的部分内容复制到新节点etcd的配置文件
+
+例:的/etc/systemd/system/etcd.service配置文件
+
+```
+[Unit]
+Description=etcd key-value store
+After=network.target
+
+[Service]
+Type=notify
+Environment=ETCD_NAME=
+Environment=ETCD_DATA_DIR=/var/lib/etcd
+Environment=ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
+Environment=ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
+Environment=ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
+Environment=ETCD_ADVERTISE_CLIENT_URLS=http://:2379
+Environment=ETCD_INITIAL_CLUSTER="=http://:2380,node182=http://:2380"
+Environment=ETCD_INITIAL_CLUSTER_STATE="existing"
+Environment=ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
+ExecStart=/usr/local/bin/etcd
+Restart=on-failure
+RestartSec=5
+User=root
+
+
+[Install]
+WantedBy=multi-user.target
+```
+
+##### 1.2.2.5 在新节点的机器上启动服务
+
+```bash
+systemctl stop etcd
+rm -rf /var/lib/etcd
+systemctl daemon-reload
+systemctl start etcd
+```
+
+##### 1.2.2.6 查看集群
+
+```bash
+etcdctl --endpoints=http://:2379 member list --write-out=table
+```
+
+##### 1.2.2.7 重复3-6步骤添加新节点进集群
+
+### 1.3 其他
+
+#### 1.3.1 删除节点
+
+查看当前集群成员列表
+
+```bash
+etcdctl member list
+```
+
+删除指定ID的节点
+
+```bash
+etcdctl member remove
+```
+
+#### 1.3.2 查看etcd集群的Leader
+
+```bash
+etcdctl get mooncake-store/mooncake_cluster/master_view --print-value-only
+```
+
+
+
+## 2 MinIO(S3)环境搭建
+### 2.1 下载 MinIO 二进制文件
+
+```bash
+wget https://dl.min.io/server/minio/release/linux-amd64/minio
+```
+
+### 2.2 赋予执行权限
+
+```bash
+chmod +x minio
+```
+
+### 2.3 移动到系统路径
+
+```bash
+sudo mv minio /usr/local/bin/
+```
+
+### 2.4 创建数据目录
+
+```bash
+mkdir -p /home/minio_data
+```
+
+### 2.5 启动MinIO
+
+设置管理员用户名和密码
+
+```bash
+export MINIO_ROOT_USER=admin
+export MINIO_ROOT_PASSWORD=adminadmin
+```
+
+配置中写好所有节点的地址,在所有设备中运行
+
+```bash
+minio server \
+ --address ":9000" \
+ --console-address ":9001" \
+ http:///home/minio_data \
+ http:///home/minio_data
+```
+
+
+### 2.6 配置自启动服务(可选)
+
+创建 `/etc/systemd/system/minio.service`文件:
+
+```ini
+[Unit]
+Description=MinIO Object Storage
+Documentation=https://docs.min.io
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+User=root
+Group=root
+# 直接设置环境变量,不依赖外部文件
+Environment="MINIO_ROOT_USER=admin"
+Environment="MINIO_ROOT_PASSWORD=admin12345"
+Environment="MINIO_VOLUMES=http:///home/minio_data http:///home/minio_data"
+ExecStart=/usr/local/bin/minio server --address ":9000" --console-address ":9001" $MINIO_VOLUMES
+Restart=always
+LimitNOFILE=65536
+
+[Install]
+WantedBy=multi-user.target
+```
+
+保存后执行:
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl start minio
+sudo systemctl status minio
+```
+
+### 2.8 下载MC
+
+```bash
+# 下载 mc 客户端
+wget https://dl.min.io/client/mc/release/linux-arm64/mc
+
+# 赋予执行权限
+chmod +x mc
+
+# 移动到系统路径,方便全局使用
+sudo mv mc /usr/local/bin/
+```
+
+### 2.9 连接MinIO服务器并添加别名
+
+```bash
+mc alias set <别名> http://<你的服务器IP>:9000 <管理员用户名> <密码>
+```
+
+### 2.10 查看服务器信息
+
+```bash
+mc admin info myminio
+```
+
+### 2.11 创建桶
+
+```bash
+mc mb myminio/mooncake-snapshot
+```
+
+### 2.12 查看所有桶
+
+```bash
+mc ls myminio
+```
+
+
+## 3 Mooncake S3环境编译
+### 3.1 配置AWS-S3环境
+
+#### 3.1.1 检查环境
+
+```bash
+ls -l /usr/local/lib64/libaws-cpp-sdk-s3.so
+```
+
+或
+
+```bash
+ls -l /usr/local/lib/libaws-cpp-sdk-s3.so
+```
+
+若文件存在,则环境搭建完成
+
+#### 3.1.2 安装(AWS SDK C++)
+
+```bash
+git clone --depth 1 --recurse-submodules --shallow-submodules https://github.com/aws/aws-sdk-cpp.git
+cd aws-sdk-cpp
+mkdir build && cd build
+cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="s3" -DENABLE_TESTING=OFF ..
+make -j$(nproc)
+make install
+ldconfig
+```
+
+### 3.2 Mooncake安装
+
+```bash
+git config --global http.sslVerify false
+git clone --recurse-submodules https://github.com/<对应仓>/Mooncake.git
+cd Mooncake
+git checkout HA
+export GOPROXY="http://mirrors.aliyun.com/goproxy,direct"
+export GOINSECURE="go.etcd.io/etcd"
+export GOSUMDB="sum.golang.org"
+export CXXFLAGS="${CXXFLAGS} -w"
+export CFLAGS="${CFLAGS} -w"
+cmake -B build -DUSE_UB=ON -DBUILD_SHARED_LIBS=ON -DWITH_TE=ON -DWITH_STORE=ON -DWITH_P2P_STORE=ON -DUSE_ETCD=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_UNIT_TESTS:BOOL=OFF -DSTORE_USE_ETCD=ON -DHAVE_AWS_SDK=ON
+cmake --build build -j$(nproc)
+git config --global http.sslVerify true
+```
+
+## 4 Mooncake HA测试脚本
+### 4.1 master运行脚本
+
+```bash
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+export MOONCAKE_SNAPSHOT_LOCAL_PATH=/home/mooncake_snapshot
+export MOONCAKE_AWS_ACCESS_KEY_ID="admin"
+export MOONCAKE_AWS_SECRET_ACCESS_KEY="adminadmin"
+export MOONCAKE_AWS_REGION="us-east-1"
+export MOONCAKE_AWS_BUCKET_NAME="mooncake-snapshot"
+export MOONCAKE_AWS_S3_ENDPOINT="http://:9000"
+export MOONCAKE_MASTER_SERVICE_SNAPSHOT_TEST_SKIP_CLEANUP=1
+export MC_LOG_DIR="/home/master_log"
+
+mooncake_master \
+ --default_kv_lease_ttl=300000 \
+ --default_kv_soft_pin_ttl=300000 \
+ --metrics_port=9006 \
+ --rpc_port=50052 \
+ --ha_backend_type=etcd \
+ --etcd_endpoints="etcd://:2379;:2379" \
+ --enable_ha=true \
+ --enable_oplog=true \
+ --rpc_address= \
+ --enable_metrics_report_to_backend=true \
+ --enable-offload=true \
+ --enable_snapshot=false \
+ --enable_snapshot_restore=false \
+ --snapshot_interval_seconds=10 \
+ --snapshot_retention_count=5 \
+ --snapshot_object_store_type="s3" \
+ --v=0 \
+ --cluster_id=mooncake_cluster
+
+```
+
+### 4.2 client运行脚本
+
+```bash
+export MC_URMA_ACTIVE_PORT=0
+export MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=/home/mooncake_ssd
+export MC_STORE_CLIENT_METRIC=0
+export MC_STORE_CLIENT_METRIC_INTERVAL=3
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=1
+export MOONCAKE_OFFLOAD_BUCKET_KEYS_LIMIT=1
+export GLOG_v=1
+export MC_LOG_DIR="/home/client_log"
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+mooncake_client \
+ --host= \
+ --metadata_server="etcd://:2379;:2379" \
+ --master_server_address="etcd://:2379;:2379" \
+ --protocol=ub \
+ --device_names=bonding_dev_0 \
+ --global_segment_size=214748364080 \
+ --port=50053 \
+ --threads=32 \
+ --v=0 \
+ --enable_offload=true
+```
+
+### 4.3 数据写入脚本
+
+```bash
+export MC_STORE_CLIENT_SETUP_RETRIES=3
+export no_proxy="127.0.0.1,localhost,local,.local,192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,141.61.0.0/16"
+export MC_STORE_CLIENT_METRIC_BANDWIDTH=0
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_TRANS_MODE=RM
+
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=1
+export MC_URMA_ACTIVE_PORT=0
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+export MC_LOG_DIR="/home/w00889253/client_log"
+
+stress_cluster_bench \
+ --metadata-server='etcd://:2379;:2379' \
+ --master-server='etcd://:2379;:2379' \
+ --local-hostname=$MC_TCP_BIND_ADDRESS \
+ --global-segment-size=0 \
+ --local-buffer-size=10737418240 \
+ --device-name=bonding_dev_0 \
+ --scenario=remote_memory \ # 四个场景segment_write,segment_read, remote_memory writer/reader
+ --role=writer \
+ --num-keys=10000 \
+ --protocol=ub \
+ --verify=true \
+ --num_threads=1 \
+ --batch-size=16 \
+ --duration=0 \
+ --master_admin_port=9006 \
+ --segments="," \
+ --replica_num=1
+
+
+
+```
+
+### 4.4 数据读取脚本
+
+```bash
+export MC_STORE_CLIENT_SETUP_RETRIES=3
+export no_proxy="127.0.0.1,localhost,local,.local,192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,141.61.0.0/16"
+export MC_STORE_CLIENT_METRIC_BANDWIDTH=0
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_TRANS_MODE=RM
+
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=1
+export MC_URMA_ACTIVE_PORT=0
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+export MC_LOG_DIR="/home/w00889253/client_log"
+
+stress_cluster_bench \
+ --metadata-server='etcd://:2379;:2379' \
+ --master-server='etcd://:2379;:2379' \
+ --local-hostname=$MC_TCP_BIND_ADDRESS \
+ --global-segment-size=0 \
+ --local-buffer-size=10737418240 \
+ --device-name=bonding_dev_0 \
+ --scenario=remote_memory \
+ --role=reader \
+ --num-keys=10000 \
+ --protocol=ub \
+ --verify=true \
+ --num_threads=1 \
+ --batch-size=16 \
+ --duration=0 \
+ --master_admin_port=9006 \
+ --segments="," \
+ --replica_num=1
+
+
+```
+
+
+
+
+## 5 多副本参数配置
+### 配置说明:replica_num / nof_replica_num 与 client TTL
+
+> 代码基线:kvcache-ai/Mooncake `main`
+> 内容:① 多副本参数 `replica_num` / `nof_replica_num` 的设置位置与方式;② client TTL 的两种配置途径(摘自《故障感知延迟分析-master多久发现节点故障.md》)
+
+---
+
+### 5.1 replica_num 与 nof_replica_num
+
+#### 5.1.1 定义
+
+定义在 `mooncake-store/include/replica.h` L81-83 的 `ReplicateConfig` 结构体:
+
+```cpp
+struct ReplicateConfig {
+ size_t replica_num{1}; // 内存副本数,默认 1
+ size_t nof_replica_num{0}; // NoF(NVMe-oF SSD)副本数,默认 0
+ ...
+};
+```
+
+#### 5.1.2 设置方式(三种场景)
+
+**方式 1:C++ 代码中直接赋值**
+
+```cpp
+ReplicateConfig config;
+config.replica_num = 2; // 2 个内存副本
+config.nof_replica_num = 1; // 1 个 NoF 副本
+auto result = client->Put(key, slices, config);
+```
+
+参考:`mooncake-store/tests/task_integration_test.cpp`、`mooncake-store/tests/replica_count_verify_test.cpp`。
+
+**方式 2:Python API**
+
+通过 pybind11 暴露为可读写属性(`mooncake-integration/store/store_py.cpp` L1832-1833,`def_readwrite`):
+
+```python
+from mooncake.store import ReplicateConfig
+
+config = ReplicateConfig()
+config.replica_num = 1 # 1 个内存副本
+config.nof_replica_num = 1 # 1 个 NoF 副本
+rc = store.put(key, data, config)
+```
+
+参考:`mooncake-wheel/tests/test_replicated_distributed_object_store.py`、`scripts/test_copy_move_api.py`。
+
+**方式 3:命令行参数 / 环境变量(E2E 与 Benchmark)**
+
+| 场景 | 参数 | 默认值 |
+|---|---|---|
+| E2E client(`store_client_e2e.py`) | `--memory-replica-num` / `--nof-replica-num` | 1 / 1 |
+| E2E 脚本(`run_nof_heartbeat_tcp_e2e.sh`) | 环境变量 `CLIENT_MEMORY_REPLICA_NUM` / `CLIENT_NOF_REPLICA_NUM` | 1 / 1 |
+| Benchmark(`store_kv_bench.py`) | `--memory-replica-num` / `--nof-replica-num` | 1 / 0 |
+
+E2E 用法示例:
+
+```bash
+CLIENT_MEMORY_REPLICA_NUM=1 CLIENT_NOF_REPLICA_NUM=1 bash run_nof_heartbeat_tcp_e2e.sh
+```
+
+#### 5.1.3 这两个值如何决定副本写入模式
+
+`DetermineReplicaWriteMode`(`mooncake-store/include/replica.h` L156-165):
+
+| `replica_num` | `nof_replica_num` | 写入模式 | 语义 |
+|---|---|---|---|
+| 1 | 0 | `SINGLE_REPLICA` | 单副本 |
+| 1 | 1 | `FLEXIBLE_DUAL_REPLICA` | 灵活双副本:1 内存 + 1 NoF,best-effort,任一类型成功即可(`HasExpectedReplicaAllocation` 只检查 `memory+nof > 0`) |
+| >1 或 >1 | 任意 | `RELIABLE_MULTI_REPLICA` | 可靠多副本:必须严格凑齐(`allocated_memory == replica_num` 且 `allocated_nof == nof_replica_num`) |
+| 0 | 0 | `SINGLE_REPLICA` | 退化为单副本 |
+
+#### 5.1.4 关键约束
+
+1. **`nof_replica_num > 0` 必须 USE_NOF=ON 编译**:否则 master 的 `PutStart` 直接返回 `INVALID_PARAMS`(`master_service.cpp` L3081-3088,`#ifndef USE_NOF` 分支;测试见 `replica_count_verify_test.cpp` Test 4)。
+2. **`replica_num` 与 `nof_replica_num` 不能同时为 0**:`store_kv_bench.py` L653 有显式校验 `ValueError`。
+3. **best-effort 语义**:`nof_replica_num == 0` 时,`HasExpectedReplicaAllocation`(`master_service.cpp` L121-133)只检查 `allocated_memory > 0`,不要求严格等于 `replica_num`。实测 `replica_num=5` 而只有 4 个 segment 时,分配 4 个副本仍算成功。
+
+---
+
+### 5.2 client TTL 的两种配置
+
+#### 5.2.1 背景结论
+
+- **master 多久发现 client 故障**:默认 **9~11 秒**(= client TTL ± 1s,TTL 默认 10s)。机制是"租约到期",不是"心跳中断检测"。
+- **可配置项**:client TTL 可配置;client 心跳间隔(1s,`client_service.cpp` L3743 硬编码)和 master 检查周期(1s,`master_service.h` L1980 硬编码)不可配。
+- **配置位置**:这是 **`mooncake_master` 的启动参数,不是 client 的**。
+
+#### 5.2.2 两种配置途径
+
+**途径 1:命令行参数 `--client_ttl`**。gflag 定义(`master.cpp` L260-264),启动时 `mooncake_master --client_ttl=5` 即生效:
+
+```cpp
+DEFINE_int64(
+ client_ttl, mooncake::DEFAULT_CLIENT_LIVE_TTL_SEC,
+ "Seconds a client stays considered alive after the last heartbeat. "
+ "If this TTL elapses without a refresh, the master treats the "
+ "client as disconnected and may unmount its segments");
+```
+
+**途径 2:master 配置文件(键名 `client_live_ttl_sec`)**。master 启动时若指定了配置文件,会从中读这个键;读不到则用 gflag 的值兜底(`master.cpp` L465-467,第三个参数就是兜底值 `FLAGS_client_ttl`):
+
+```cpp
+ default_config.GetInt64("client_live_ttl_sec",
+ &master_config.client_live_ttl_sec,
+ FLAGS_client_ttl);
+```
+
+**优先级:命令行显式设置 > 配置文件**。命令行显式传了 `--client_ttl`(`!info.is_default` 判断非默认值),或根本没用配置文件(`!conf_set`),则用命令行值覆盖(`master.cpp` L949-953):
+
+```cpp
+ if ((google::GetCommandLineFlagInfo("client_ttl", &info) &&
+ !info.is_default) ||
+ !conf_set) {
+ master_config.client_live_ttl_sec = FLAGS_client_ttl;
+ }
+```
+
+#### 5.2.3 配置速查表
+
+| 参数 | 默认 | 可配 | 配置在哪 | 源码位置 |
+|---|---|---|---|---|
+| client TTL | 10s | ✅ | `mooncake_master --client_ttl=<秒>` / master 配置文件 `client_live_ttl_sec` | `master.cpp` L260-264 / L465-467 / L949-953;`types.h` L95 `DEFAULT_CLIENT_LIVE_TTL_SEC=10` |
+| client 心跳间隔 | 1s | ❌ | —(client 代码硬编码) | `client_service.cpp` L3743 |
+| master 检查周期 | 1s | ❌ | —(master 代码硬编码) | `master_service.h` L1980 |
+
+一句话:同一个参数,命令行叫 `client_ttl`、配置文件里叫 `client_live_ttl_sec`,最终都写进 `master_config.client_live_ttl_sec`,两头都设时命令行优先。
+
+实测:E2E 中 master 以 `--client_ttl=5` 启动,kill 数据节点后感知延迟落在 4~6s 区间,与理论吻合。
+
+### 5.3 bench方案验证
+
+验证思路:验证内存存在副本和disk磁盘存在副本两种方式进行验证。
+
+#### 5.3.1 内存存在副本验证思路
+
+写入500个key,如果存在两个client,则应该两边分别写入250个key,同时应该在另外一个client上面存在副本,则每个节点client内存应该有500个key。
+操作步骤:先写入500个key,然后断连一个节点client,读取全部的key,可以从另外一个节点上全部读取。
+
+#### 5.3.2 disk磁盘存在副本验证思路
+
+写入500个key,如果存在两个client,则两边都存在250个key,,同时应该在另外一个client上面存在副本,则每个节点client内存应该有500个key。同时将这些key卸载到本地磁盘当中,磁盘当中应该在每个节点上有完整的500个key。
+
+操作步骤:先写入500个key,然后断连两个节点client, 重启其中一个client,此时磁盘中的卸载的数据可以在master上重新加载,有500个key,此时读取数据应该能够将所有的数据都读到。
+
+参考脚本:
+master
+
+```
+mooncake_master \
+ --enable_http_metadata_server=true \
+ --http_metadata_server_host=0.0.0.0 \
+ --http_metadata_server_port=9109 \
+ --default_kv_lease_ttl=300000 \
+ --default_kv_soft_pin_ttl=300000 \
+ --metrics_port=9006 \
+ --rpc_port=50052 \
+ --enable-offload=true
+
+```
+
+client
+
+```
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_ACTIVE_PORT=0
+export MOONCAKE_OFFLOAD_FILE_STORAGE_PATH=/home/mooncake_ssd
+export MC_STORE_CLIENT_METRIC=0
+export MC_STORE_CLIENT_METRIC_INTERVAL=3
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=0
+export MOONCAKE_OFFLOAD_BUCKET_KEYS_LIMIT=1(重要,确保所有key存入到磁盘当中)
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+mooncake_client \
+ --host= \
+ --metadata_server=http://:9109/metadata \
+ --master_server_address=:50052 \
+ --protocol=ub \
+ --device_names=bonding_dev_0 \
+ --global_segment_size=21474836408 \
+ --port=50053 \
+ --threads=32 \
+ --v=0 \
+ --enable_offload=true
+
+```
+
+写入/读取脚本
+
+```
+export MC_STORE_CLIENT_SETUP_RETRIES=3
+export no_proxy="127.0.0.1,localhost,local,.local,192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,141.61.0.0/16"
+export MC_STORE_CLIENT_METRIC_BANDWIDTH=0
+export MC_TCP_BIND_ADDRESS=
+export MC_URMA_TRANS_MODE=RM
+export MC_LOG_ENABLE=off
+# export MC_LOG_DIR=/home/log
+export MC_URMA_BONDING_MULTIPATH_ENABLE=on
+export MC_HIFREQ_LOG_SAMPLE_RATE=0
+export MC_URMA_ACTIVE_PORT=0
+
+export URMA_RPC_ENABLE=0
+export URMA_RPC_DEVICE=bonding_dev_0
+export URMA_RPC_EID_INDEX=0
+
+stress_cluster_bench \
+ --metadata-server='http://:9109/metadata' \
+ --master-server=':50052' \
+ --local-hostname=$MC_TCP_BIND_ADDRESS \
+ --global-segment-size=0 \
+ --local-buffer-size=10737418240 \
+ --device-name=bonding_dev_0 \
+ --scenario=remote_memory \ (使用remote memory,这样key可以以固定结构写入)
+ --role=reader \ (写入数据使用writer,读取使用reader)
+ --num-keys=500 \
+ --protocol=ub \
+ --verify=false \
+ --num_threads=1 \
+ --batch-size=16 \
+ --duration=0 \
+ --master_admin_port=9006 \
+ --replica_num=2 (副本数量)
+
+```
+
+
+# 6 mooncake基础信息上报
+
+在上述启动master的脚本中,配置```enable_metrics_report_to_backend=true``,可以通过etcd命令行查找到对应数据
+参考示例如下:
+
+```
+[root@node1 Mooncake]# etcdctl --endpoints=:2379,:2379 get --prefix /mooncake_cluster/masters/primary
+/mooncake_cluster/masters/primary
+{"id":"4056379308262733915-15491753023959092137","hostname":":50052","role":"primary","mem_total_bytes":42949672816,"mem_used_bytes":4194304000,"mem_available_bytes":38755368816,"nof_total_bytes":0,"nof_used_bytes":0,"nof_available_bytes":0,"file_total_bytes":4398046511104,"file_used_bytes":9080668160,"file_available_bytes":4388965842944,"key_count":500,"active_clients":2,"updated_at":"2026-08-06T16:59:35+08:00"}
+[root@node1 Mooncake]# etcdctl --endpoints=:2379,:2379 get --prefix /mooncake_cluster/masters/primary
+/mooncake_cluster/masters/primary
+{"id":"4056379308262733915-15491753023959092137","hostname":":50052","role":"primary","mem_total_bytes":42949672816,"mem_used_bytes":4194304000,"mem_available_bytes":38755368816,"nof_total_bytes":0,"nof_used_bytes":0,"nof_available_bytes":0,"file_total_bytes":4398046511104,"file_used_bytes":9080668160,"file_available_bytes":4388965842944,"key_count":500,"active_clients":2,"updated_at":"2026-08-06T17:00:40+08:00"}
+```
+
+
+# 7 mooncake性能信息
+
+使用脚本```Mooncake/mooncake-store/benchmarks/cluster_mooncake_diag.py```
+参考其中的使用方法,在对应脚本当中设置对应log日志,则可以进行读取。
\ No newline at end of file
diff --git a/mooncake-common/etcd/etcd_wrapper.go b/mooncake-common/etcd/etcd_wrapper.go
index 7cedada048..f75a1e3415 100644
--- a/mooncake-common/etcd/etcd_wrapper.go
+++ b/mooncake-common/etcd/etcd_wrapper.go
@@ -811,8 +811,49 @@ func EtcdStoreBatchCreateWrapper(keys **C.char, values **C.char, count C.int, er
return 0
}
+//export EtcdStoreBatchPutWithLeaseWrapper
+func EtcdStoreBatchPutWithLeaseWrapper(keys **C.char, keySizes *C.int, values **C.char, valueSizes *C.int, count C.int, leaseId int64, errMsg **C.char) int {
+ cli := getStoreClient()
+ if cli == nil {
+ *errMsg = C.CString("etcd client not initialized")
+ return -1
+ }
+
+ n := int(count)
+ if n == 0 {
+ return 0
+ }
+
+ // Unsafe casting to access C arrays as Go slices; key/value buffers are
+ // binary-safe (carry explicit sizes).
+ keyPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(keys))[:n:n]
+ keySizeList := (*[1 << 28]C.int)(unsafe.Pointer(keySizes))[:n:n]
+ valPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(values))[:n:n]
+ valSizeList := (*[1 << 28]C.int)(unsafe.Pointer(valueSizes))[:n:n]
+
+ ops := make([]clientv3.Op, 0, n)
+ for i := 0; i < n; i++ {
+ k := C.GoStringN(keyPtrs[i], keySizeList[i])
+ v := C.GoStringN(valPtrs[i], valSizeList[i])
+ // Bind all keys to the single master lease. Caller is responsible for
+ // fencing (the keys must already be absent / owned by self); this is an
+ // unconditional atomic batch, so it can also be reused for reaffirm.
+ ops = append(ops, clientv3.OpPut(k, v, clientv3.WithLease(clientv3.LeaseID(leaseId))))
+ }
+
+ ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
+ defer cancel()
+
+ _, err := cli.Txn(ctx).Then(ops...).Commit()
+ if err != nil {
+ *errMsg = C.CString(err.Error())
+ return -1
+ }
+ return 0
+}
+
//export EtcdStoreTxnCompareAndPutWrapper
-func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.int, compareKinds *C.int, compareValues **C.char, compareValueSizes *C.int, compareCount C.int, putKeys **C.char, putKeySizes *C.int, putValues **C.char, putValueSizes *C.int, putCount C.int, errMsg **C.char) int {
+func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.int, compareKinds *C.int, compareValues **C.char, compareValueSizes *C.int, compareCount C.int, putKeys **C.char, putKeySizes *C.int, putValues **C.char, putValueSizes *C.int, putCount C.int, deleteKeys **C.char, deleteKeySizes *C.int, deleteCount C.int, errMsg **C.char) int {
cli := getStoreClient()
if cli == nil {
*errMsg = C.CString("etcd client not initialized")
@@ -821,6 +862,7 @@ func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.i
cmpN := int(compareCount)
putN := int(putCount)
+ deleteN := int(deleteCount)
cmps := make([]clientv3.Cmp, 0, cmpN)
if cmpN > 0 {
@@ -844,7 +886,7 @@ func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.i
}
}
- ops := make([]clientv3.Op, 0, putN)
+ ops := make([]clientv3.Op, 0, putN+deleteN)
if putN > 0 {
putKeyPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(putKeys))[:putN:putN]
putKeySizeList := (*[1 << 28]C.int)(unsafe.Pointer(putKeySizes))[:putN:putN]
@@ -856,6 +898,14 @@ func EtcdStoreTxnCompareAndPutWrapper(compareKeys **C.char, compareKeySizes *C.i
ops = append(ops, clientv3.OpPut(k, v))
}
}
+ if deleteN > 0 {
+ deleteKeyPtrs := (*[1 << 28]*C.char)(unsafe.Pointer(deleteKeys))[:deleteN:deleteN]
+ deleteKeySizeList := (*[1 << 28]C.int)(unsafe.Pointer(deleteKeySizes))[:deleteN:deleteN]
+ for i := 0; i < deleteN; i++ {
+ k := C.GoStringN(deleteKeyPtrs[i], deleteKeySizeList[i])
+ ops = append(ops, clientv3.OpDelete(k))
+ }
+ }
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
diff --git a/mooncake-common/include/rpc_client_io_context.h b/mooncake-common/include/rpc_client_io_context.h
index b9842f0c99..fb575df939 100644
--- a/mooncake-common/include/rpc_client_io_context.h
+++ b/mooncake-common/include/rpc_client_io_context.h
@@ -8,6 +8,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,9 +27,10 @@ coro_io::io_context_pool& GetRpcClientIoContextPool(uint32_t thread_count) {
}
/**
- * A replaceable client pool for callers that communicate with one target at a
- * time. Requests retain a shared_ptr to the old pool while they are in flight;
- * after an address switch the old pool is destroyed when those requests end.
+ * A client pool accessor that caches one client pool per target address, so
+ * callers that alternate between several targets (e.g. submaster routing)
+ * reuse existing connections instead of recreating a pool on every switch.
+ * The most recently selected pool is also exposed via GetClientPool().
*/
class RpcClientPool {
public:
@@ -38,18 +40,22 @@ class RpcClientPool {
explicit RpcClientPool(coro_io::io_context_pool& io_context_pool,
PoolConfig config = {})
: io_context_pool_(io_context_pool), config_(std::move(config)) {
- // Address replacement supersedes background recovery of the old host.
+ // Explicit target selection supersedes background recovery of an old
+ // host; the pool's own connect/retry still applies per request.
config_.host_alive_detect_duration = std::chrono::seconds(0);
}
std::shared_ptr GetOrCreateClientPool(
std::string_view address) {
std::lock_guard lock(mutex_);
- if (!client_pool_ || address_ != address) {
- client_pool_ =
- ClientPool::create(address, config_, io_context_pool_);
- address_ = address;
+ std::string addr(address);
+ auto it = pools_.find(addr);
+ if (it == pools_.end()) {
+ auto pool = ClientPool::create(addr, config_, io_context_pool_);
+ it = pools_.emplace(addr, std::move(pool)).first;
}
+ address_ = std::move(addr);
+ client_pool_ = it->second;
return client_pool_;
}
@@ -58,12 +64,21 @@ class RpcClientPool {
return client_pool_;
}
+ std::string GetAddress() const {
+ std::shared_lock lock(mutex_);
+ return address_;
+ }
+
private:
mutable std::shared_mutex mutex_;
coro_io::io_context_pool& io_context_pool_;
PoolConfig config_;
std::string address_;
std::shared_ptr client_pool_;
+ // Cache of client pools keyed by target address, so switching back and
+ // forth between submaster addresses reuses existing connections instead of
+ // recreating the pool on every switch.
+ std::unordered_map> pools_;
};
} // namespace mooncake
diff --git a/mooncake-common/tests/rpc_client_io_context_test.cpp b/mooncake-common/tests/rpc_client_io_context_test.cpp
index c3b18c7127..d63fd5e34e 100644
--- a/mooncake-common/tests/rpc_client_io_context_test.cpp
+++ b/mooncake-common/tests/rpc_client_io_context_test.cpp
@@ -41,18 +41,19 @@ TEST(RpcClientIoContextPoolTest, UsesConfiguredSizeAndReusesPool) {
EXPECT_NE(&first_pool, &second_pool);
}
-TEST(RpcClientIoContextPoolTest, ReplacesPoolWhenTargetChanges) {
+TEST(RpcClientIoContextPoolTest, CachesPoolsPerAddress) {
RpcClientPool pools(GetFirstTestRpcClientIoContextPool());
auto first = pools.GetOrCreateClientPool("127.0.0.1:10001");
- std::weak_ptr old_pool = first;
EXPECT_EQ(pools.GetOrCreateClientPool("127.0.0.1:10001"), first);
auto second = pools.GetOrCreateClientPool("127.0.0.1:10002");
EXPECT_NE(first, second);
- first.reset();
- EXPECT_TRUE(old_pool.expired());
EXPECT_EQ(pools.GetClientPool(), second);
+
+ // Switching back reuses the cached pool instead of recreating it.
+ EXPECT_EQ(pools.GetOrCreateClientPool("127.0.0.1:10001"), first);
+ EXPECT_EQ(pools.GetClientPool(), first);
}
TEST(RpcClientIoContextPoolTest, SendsToNewAddressAfterSwitch) {
diff --git a/mooncake-store/benchmarks/CMakeLists.txt b/mooncake-store/benchmarks/CMakeLists.txt
index 817230b940..b30d7cba8f 100644
--- a/mooncake-store/benchmarks/CMakeLists.txt
+++ b/mooncake-store/benchmarks/CMakeLists.txt
@@ -38,6 +38,13 @@ target_link_libraries(
stress_cluster_bench PRIVATE mooncake_store transfer_engine asio_shared
gflags::gflags glog::glog pthread)
+# CFM client benchmark. Simulates vLLM inference requests as KV-cache block
+# accesses and reports both client performance and IO Pattern observability.
+add_executable(cfm_client_bench cfm_client_bench.cpp)
+target_link_libraries(
+ cfm_client_bench PRIVATE mooncake_store transfer_engine asio_shared
+ gflags::gflags glog::glog pthread)
+
# Benchmark for vLLM Store Connector path
# Triggers: batch_put_from_multi_buffers / batchIsExist /
# batch_get_into_multi_buffers (and setup/register_buffer/tearDownAll).
@@ -75,3 +82,10 @@ if(STORE_USE_ETCD)
oplog_batch_bench PRIVATE mooncake_store glog::glog gflags::gflags
JsonCpp::JsonCpp)
endif()
+
+if(STORE_USE_ETCD)
+ add_executable(vchunk_distributed_bench vchunk_distributed_bench.cpp)
+ target_link_libraries(
+ vchunk_distributed_bench PRIVATE mooncake_store transfer_engine asio_shared
+ gflags::gflags glog::glog pthread)
+endif()
diff --git a/mooncake-store/benchmarks/cfm_client_bench.cpp b/mooncake-store/benchmarks/cfm_client_bench.cpp
new file mode 100644
index 0000000000..41db1a1a04
--- /dev/null
+++ b/mooncake-store/benchmarks/cfm_client_bench.cpp
@@ -0,0 +1,533 @@
+// CFM client benchmark that models the vLLM KV-cache call path.
+//
+// One benchmark request consists of prompt_tokens + output_tokens. The KV
+// cache is split into tokens_per_block blocks for every transformer layer,
+// exactly as a vLLM connector would address its layer/block cache entries.
+// Each request records inference and access metrics, reports a snapshot through
+// CfmClientImpl, then prints both CFM call latency and IO Pattern metrics.
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "gflags/gflags.h"
+#include "glog/logging.h"
+#include "io_pattern/cfm_client_impl.h"
+#include "io_pattern/cfm_protocol.h"
+#include "io_pattern/cfm_service.h"
+#include "io_pattern/rpc_transport.h"
+#include "io_pattern/runtime.h"
+
+namespace {
+
+using Clock = std::chrono::steady_clock;
+using mooncake::ErrorCode;
+using mooncake::TenantId;
+using namespace mooncake::io_pattern;
+
+DEFINE_uint64(requests, 20, "Number of vLLM-style inference requests");
+DEFINE_uint64(prompt_tokens, 1024, "Input tokens in each inference request");
+DEFINE_uint64(output_tokens, 128, "Decode tokens in each inference request");
+DEFINE_uint64(tokens_per_block, 16, "Tokens represented by one KV block");
+DEFINE_uint64(num_layers, 32, "Transformer layers represented per request");
+DEFINE_uint64(kv_block_bytes, 256 * 1024,
+ "Bytes in one layer/block KV-cache object");
+DEFINE_uint64(num_sessions, 4, "Independent vLLM request sessions");
+DEFINE_uint64(shared_prefix_tokens, 512,
+ "Per-session prompt prefix reused by later requests");
+DEFINE_uint64(report_capacity, 262144,
+ "Maximum queued IO Pattern observations before reporting");
+DEFINE_uint64(policy_queue_capacity, 4096,
+ "Maximum CFM policy commands queued for the client");
+DEFINE_uint64(report_flush_wait_ms, 1100,
+ "Maximum wait for remote CFM policy production after a flush");
+DEFINE_double(memory_used_ratio, 0.95,
+ "Reported L1 memory use ratio; >= 0.90 triggers CFM planning");
+DEFINE_string(tenant, "vllm-benchmark", "Tenant id");
+DEFINE_string(node_id, "vllm-submaster-0", "CFM node/submaster id");
+DEFINE_string(cfm_endpoint, "",
+ "Remote CFM coro_rpc endpoint; empty uses an embedded CFM service");
+DEFINE_string(cfm_auth_token, "cfm-client-benchmark-node-token",
+ "Authentication token for the CFM node endpoint");
+
+uint64_t SteadyNowNs() {
+ return static_cast(
+ std::chrono::duration_cast(
+ Clock::now().time_since_epoch())
+ .count());
+}
+
+double ToMicroseconds(Clock::duration duration) {
+ return std::chrono::duration(duration).count();
+}
+
+size_t BlockCount(uint64_t tokens) {
+ return static_cast((tokens + FLAGS_tokens_per_block - 1) /
+ FLAGS_tokens_per_block);
+}
+
+std::string KvKey(size_t session, size_t request, size_t layer, size_t block,
+ bool is_shared_prefix) {
+ const auto owner = is_shared_prefix ? std::string("prefix")
+ : std::string("request-") +
+ std::to_string(request);
+ return "vllm/" + FLAGS_node_id + "/session-" +
+ std::to_string(session) + "/" + owner + "/layer-" +
+ std::to_string(layer) + "/block-" + std::to_string(block);
+}
+
+class ServiceBackedCfmTransport final : public CfmRpcTransport {
+ public:
+ ServiceBackedCfmTransport(std::shared_ptr service,
+ std::string node_id)
+ : service_(std::move(service)), node_id_(std::move(node_id)) {}
+
+ bool Authenticate(std::string_view token) override {
+ if (!service_ || !service_->AuthenticateNode(token)) return false;
+ token_ = std::string(token);
+ authenticated_ = true;
+ return true;
+ }
+
+ bool Send(std::string_view method, std::string_view payload,
+ std::chrono::milliseconds) override {
+ return authenticated_ && service_ &&
+ service_->Send(node_id_, method, payload, token_);
+ }
+
+ CfmReceiveResult Receive(std::string_view method,
+ std::chrono::milliseconds) override {
+ if (!authenticated_ || !service_ || method != "poll_policy") {
+ return CfmReceiveResult::Error();
+ }
+ const auto delivery = service_->PollPolicy(node_id_, token_);
+ return delivery
+ ? CfmReceiveResult::Payload(delivery->second, delivery->first)
+ : CfmReceiveResult::Empty();
+ }
+
+ bool Acknowledge(uint64_t delivery_id, bool success,
+ std::chrono::milliseconds) override {
+ return authenticated_ && service_ &&
+ service_->AcknowledgePolicy(node_id_, delivery_id, success,
+ token_);
+ }
+
+ private:
+ std::shared_ptr service_;
+ std::string node_id_;
+ std::string token_;
+ bool authenticated_{false};
+};
+
+class LatencyStats final {
+ public:
+ void Record(double value_us) { values_us_.push_back(value_us); }
+
+ double Percentile(double percentile) const {
+ if (values_us_.empty()) return 0.0;
+ const double rank = percentile / 100.0 * (values_us_.size() - 1);
+ const auto lower = static_cast(rank);
+ const auto upper = std::min(lower + 1, values_us_.size() - 1);
+ const double fraction = rank - lower;
+ return values_us_[lower] * (1.0 - fraction) +
+ values_us_[upper] * fraction;
+ }
+
+ void Finalize() { std::sort(values_us_.begin(), values_us_.end()); }
+
+ double Mean() const {
+ if (values_us_.empty()) return 0.0;
+ return std::accumulate(values_us_.begin(), values_us_.end(), 0.0) /
+ values_us_.size();
+ }
+
+ private:
+ std::vector values_us_;
+};
+
+struct MetricReportSnapshot {
+ uint64_t calls{0};
+ uint64_t failures{0};
+ uint64_t observations{0};
+ LatencyStats latency;
+};
+
+class MetricReportStats final {
+ public:
+ void Record(const MetricBatch& batch, double latency_us, bool success) {
+ std::lock_guard lock(mutex_);
+ ++calls;
+ if (!success) ++failures;
+ observations +=
+ batch.inference.size() + batch.accesses.size() + batch.storage.size();
+ latency.Record(latency_us);
+ }
+
+ MetricReportSnapshot Finalize() {
+ std::lock_guard lock(mutex_);
+ latency.Finalize();
+ return {.calls = calls,
+ .failures = failures,
+ .observations = observations,
+ .latency = latency};
+ }
+
+ private:
+ std::mutex mutex_;
+ uint64_t calls{0};
+ uint64_t failures{0};
+ uint64_t observations{0};
+ LatencyStats latency;
+};
+
+struct RequestData {
+ IoPatternSnapshot snapshot;
+ std::vector inference;
+ std::vector accesses;
+};
+
+RequestData BuildRequest(size_t request_index) {
+ const size_t session = request_index % FLAGS_num_sessions;
+ const uint64_t total_tokens = FLAGS_prompt_tokens + FLAGS_output_tokens;
+ const size_t blocks = BlockCount(total_tokens);
+ const size_t shared_blocks =
+ std::min(blocks, BlockCount(FLAGS_shared_prefix_tokens));
+ const bool prefix_is_cached = request_index >= FLAGS_num_sessions;
+ const uint64_t now_ns = SteadyNowNs();
+
+ RequestData request;
+ request.snapshot.generated_at_ns = now_ns;
+ request.inference.reserve(blocks * FLAGS_num_layers);
+ request.accesses.reserve(blocks * FLAGS_num_layers);
+ request.snapshot.keys.reserve(blocks * FLAGS_num_layers);
+ const auto tenant = TenantId(FLAGS_tenant);
+ const auto session_id = "vllm-session-" + std::to_string(session);
+
+ for (size_t layer = 0; layer < FLAGS_num_layers; ++layer) {
+ for (size_t block = 0; block < blocks; ++block) {
+ const bool is_shared_prefix = block < shared_blocks;
+ const bool is_hit = is_shared_prefix && prefix_is_cached;
+ const ObjectRef object{
+ .tenant_id = tenant,
+ .key = KvKey(session, request_index, layer, block,
+ is_shared_prefix)};
+ const auto block_end = std::min(
+ total_tokens, (block + 1) * FLAGS_tokens_per_block);
+ const auto block_tokens = static_cast(
+ block_end - block * FLAGS_tokens_per_block);
+
+ InferenceMetrics inference{
+ .object = object,
+ .session_id = session_id,
+ .layout = CacheLayout::kLayerFirst,
+ .layout_group = static_cast(layer),
+ .prefix_depth = static_cast(shared_blocks),
+ .prefix_fanout = static_cast(FLAGS_num_sessions),
+ .match_length = is_hit
+ ? static_cast(
+ FLAGS_shared_prefix_tokens)
+ : 0U,
+ .continuous_prefix_length = is_hit
+ ? static_cast(
+ FLAGS_shared_prefix_tokens)
+ : 0U,
+ .token_count = block_tokens,
+ .recompute_cost = is_hit ? 0.0F : static_cast(block_tokens),
+ .request_priority = 1};
+ AccessRecord access{
+ .object = object,
+ .observed_at_ns = now_ns,
+ .block_size = FLAGS_kv_block_bytes,
+ .latency_us = is_hit ? 20U : 200U,
+ .tier = CacheTier::kL1Host,
+ .operation = is_hit ? IoOperation::kGet : IoOperation::kPut,
+ .is_hit = is_hit,
+ .write_batch_size = is_hit
+ ? 0U
+ : static_cast(FLAGS_num_layers),
+ .overwrite = !is_hit && is_shared_prefix};
+ request.inference.push_back(inference);
+ request.accesses.push_back(access);
+ request.snapshot.keys.push_back(
+ KeyMetrics{.object = object,
+ .session_id = session_id,
+ .last_access_time_ns = now_ns,
+ .access_count_window = 1,
+ .block_size = FLAGS_kv_block_bytes,
+ .token_count = block_tokens,
+ .prefix_depth = static_cast(shared_blocks),
+ .prefix_fanout = static_cast(FLAGS_num_sessions),
+ .match_length = inference.match_length,
+ .continuous_prefix_length =
+ inference.continuous_prefix_length,
+ .write_batch_size = access.write_batch_size,
+ .write_frequency =
+ access.operation == IoOperation::kPut ? 1U : 0U,
+ .write_object_size = FLAGS_kv_block_bytes,
+ .recompute_cost = inference.recompute_cost,
+ .overwrite_ratio = access.overwrite ? 1.0F : 0.0F,
+ .replica_tiers = CacheTierBit(CacheTier::kL1Host),
+ .layout = CacheLayout::kLayerFirst,
+ .layout_group = static_cast(layer),
+ .request_priority = 1,
+ .active = is_hit,
+ .write_burst = !is_hit});
+ }
+ }
+ request.snapshot.storage.push_back(
+ StorageMetric{.source_id = FLAGS_node_id,
+ .observed_at_ns = now_ns,
+ .tier = CacheTier::kL1Host,
+ .read_bandwidth_bytes_per_sec = 20ULL * 1024 * 1024 * 1024,
+ .write_bandwidth_bytes_per_sec = 10ULL * 1024 * 1024 * 1024,
+ .read_latency_us = 20,
+ .write_latency_us = 200,
+ .used_bytes = static_cast(
+ FLAGS_memory_used_ratio * 1024 * 1024 * 1024),
+ .capacity_bytes = 1024ULL * 1024 * 1024,
+ .rpc_latency_us = 100,
+ .memory_used_ratio =
+ static_cast(FLAGS_memory_used_ratio)});
+ return request;
+}
+
+void PrintObservability(std::string_view name,
+ const IoPatternObservabilitySnapshot& metrics) {
+ std::cout << "\n " << name << " IO Pattern metrics\n"
+ << " collect max latency: " << metrics.collect_latency_us
+ << " us\n"
+ << " analyze max latency: " << metrics.analyze_latency_us
+ << " us\n"
+ << " policy decisions: " << metrics.policy_decisions
+ << " (" << std::fixed << std::setprecision(2)
+ << metrics.policy_decision_qps << " qps)\n"
+ << " strategy hit rate: " << metrics.strategy_hit_rate * 100
+ << "%\n"
+ << " false positive rate: "
+ << metrics.false_positive_rate * 100 << "%\n"
+ << " degraded: " << metrics.degrade_count << "\n"
+ << " report drops: " << metrics.report_drop_count
+ << "\n";
+}
+
+bool ValidateFlags() {
+ return FLAGS_requests != 0 && FLAGS_prompt_tokens + FLAGS_output_tokens != 0 &&
+ FLAGS_tokens_per_block != 0 && FLAGS_num_layers != 0 &&
+ FLAGS_kv_block_bytes != 0 && FLAGS_num_sessions != 0 &&
+ FLAGS_report_capacity != 0 && FLAGS_policy_queue_capacity != 0 &&
+ FLAGS_memory_used_ratio >= 0.0 && FLAGS_memory_used_ratio <= 1.0;
+}
+
+} // namespace
+
+int main(int argc, char* argv[]) {
+ google::InitGoogleLogging(argv[0]);
+ gflags::ParseCommandLineFlags(&argc, &argv, true);
+ if (!ValidateFlags()) {
+ LOG(ERROR) << "All numeric size/count flags must be positive and "
+ "--memory_used_ratio must be within [0, 1]";
+ return 1;
+ }
+
+ constexpr char kProducerToken[] = "cfm-client-benchmark-producer-token";
+ std::atomic eviction_commands{0};
+ std::atomic prefetch_commands{0};
+ std::atomic admission_commands{0};
+
+ std::shared_ptr embedded_service;
+ std::shared_ptr transport;
+ if (FLAGS_cfm_endpoint.empty()) {
+ auto cfm_control_runtime = std::make_shared(
+ IoPatternRuntime::Handlers{
+ .eviction = [](const EvictionPlan&) { return ErrorCode::OK; },
+ .prefetch = [](const PrefetchPlan&) { return ErrorCode::OK; },
+ .admission = [](const AdmissionResult&) {
+ return ErrorCode::OK;
+ }});
+ embedded_service = std::make_shared(
+ cfm_control_runtime, FLAGS_cfm_auth_token,
+ FLAGS_policy_queue_capacity, kProducerToken);
+ transport = std::make_shared(
+ embedded_service, FLAGS_node_id);
+ } else {
+ transport = std::make_shared(
+ FLAGS_cfm_endpoint, FLAGS_node_id, std::chrono::milliseconds(500));
+ }
+ auto codec = std::make_shared();
+ auto channel = std::make_shared(
+ transport, codec,
+ CfmRpcConfig{.timeout = std::chrono::milliseconds(500),
+ .auth_token = FLAGS_cfm_auth_token});
+
+ IoPatternRuntime::Config source_config;
+ source_config.report_capacity = FLAGS_report_capacity;
+ MetricReportStats metric_reports;
+ source_config.report_sink = [&channel, &metric_reports](const MetricBatch& batch) {
+ const auto started = Clock::now();
+ const bool success = channel->SendMetricBatch(batch);
+ metric_reports.Record(batch, ToMicroseconds(Clock::now() - started),
+ success);
+ return success;
+ };
+ auto source_runtime = std::make_shared(
+ IoPatternRuntime::Handlers{
+ .eviction = [&eviction_commands](const EvictionPlan&) {
+ ++eviction_commands;
+ return ErrorCode::OK;
+ },
+ .prefetch = [&prefetch_commands](const PrefetchPlan&) {
+ ++prefetch_commands;
+ return ErrorCode::OK;
+ },
+ .admission = [&admission_commands](const AdmissionResult&) {
+ ++admission_commands;
+ return ErrorCode::OK;
+ }},
+ source_config);
+ CfmClientImpl client(channel, [&source_runtime](const PolicyCommand& command) {
+ return source_runtime->ExecuteCommand(command);
+ });
+
+ LatencyStats report_latency;
+ uint64_t failed_reports = 0;
+ uint64_t total_blocks = 0;
+ const auto benchmark_start = Clock::now();
+ for (size_t request_index = 0; request_index < FLAGS_requests;
+ ++request_index) {
+ auto request = BuildRequest(request_index);
+ total_blocks += request.accesses.size();
+ for (size_t i = 0; i < request.inference.size(); ++i) {
+ source_runtime->ReportInferenceMetrics(request.inference[i]);
+ source_runtime->RecordAccess(request.accesses[i].object.key,
+ request.accesses[i]);
+ }
+ source_runtime->RecordStorageMetric(request.snapshot.storage.front());
+
+ const auto report_start = Clock::now();
+ const auto result = client.ReportSnapshot(request.snapshot);
+ report_latency.Record(ToMicroseconds(Clock::now() - report_start));
+ if (result != ErrorCode::OK) ++failed_reports;
+ }
+ const auto submission_seconds =
+ std::chrono::duration(Clock::now() - benchmark_start).count();
+
+ // Stop joins the reporter worker and performs its final flush. No new
+ // metric batch can reach CFM after this returns.
+ source_runtime->StopReports();
+ if (embedded_service) {
+ if (!embedded_service->WaitForPolicyIdle(
+ std::chrono::milliseconds(FLAGS_report_flush_wait_ms))) {
+ LOG(WARNING) << "Timed out waiting for embedded CFM policy production";
+ }
+ } else {
+ std::this_thread::sleep_for(
+ std::chrono::milliseconds(FLAGS_report_flush_wait_ms));
+ }
+ uint64_t observed_commands = 0;
+ bool policy_drain_complete = false;
+ const auto policy_deadline =
+ Clock::now() + std::chrono::milliseconds(FLAGS_report_flush_wait_ms);
+ while (Clock::now() < policy_deadline) {
+ if (client.PollAndDispatchPolicy() != ErrorCode::OK) break;
+ const uint64_t executed = eviction_commands + prefetch_commands +
+ admission_commands;
+ if (executed == observed_commands) {
+ policy_drain_complete = embedded_service != nullptr;
+ break;
+ } else {
+ observed_commands = executed;
+ }
+ std::this_thread::sleep_for(std::chrono::milliseconds(1));
+ }
+
+ const auto end_to_end_seconds =
+ std::chrono::duration(Clock::now() - benchmark_start).count();
+
+ const auto source_snapshot = source_runtime->Snapshot();
+ const auto source_metrics =
+ source_runtime->ObservabilitySnapshot(end_to_end_seconds);
+ source_runtime.reset();
+ report_latency.Finalize();
+ const auto metric_report_snapshot = metric_reports.Finalize();
+ const auto cfm_snapshot = embedded_service
+ ? embedded_service->SnapshotForNode(FLAGS_node_id)
+ : IoPatternSnapshot{};
+ const auto cfm_metrics = embedded_service
+ ? embedded_service->ObservabilityForNode(
+ FLAGS_node_id, end_to_end_seconds)
+ : IoPatternObservabilitySnapshot{};
+
+ std::cout << "\n============================================================\n"
+ << "CFM CLIENT BENCHMARK (vLLM inference request model)\n"
+ << "============================================================\n"
+ << " Requests: " << FLAGS_requests << "\n"
+ << " Tokens/request: "
+ << FLAGS_prompt_tokens + FLAGS_output_tokens << " (prompt="
+ << FLAGS_prompt_tokens << ", decode=" << FLAGS_output_tokens
+ << ")\n"
+ << " KV blocks/request: " << BlockCount(
+ FLAGS_prompt_tokens + FLAGS_output_tokens) * FLAGS_num_layers
+ << " (layers=" << FLAGS_num_layers << ")\n"
+ << " Total KV blocks: " << total_blocks << "\n"
+ << " Request submission time: " << std::fixed
+ << std::setprecision(2) << submission_seconds << " s\n"
+ << " Submission requests/sec: "
+ << FLAGS_requests / submission_seconds << "\n"
+ << " End-to-end time: " << end_to_end_seconds << " s\n"
+ << "\n CFM ReportSnapshot latency\n"
+ << " failed reports: " << failed_reports << "\n"
+ << " mean: " << report_latency.Mean() << " us\n"
+ << " p50 / p90 / p99: " << report_latency.Percentile(50)
+ << " / " << report_latency.Percentile(90) << " / "
+ << report_latency.Percentile(99) << " us\n"
+ << "\n CFM report_metric_batch latency\n"
+ << " calls / failures: " << metric_report_snapshot.calls
+ << " / " << metric_report_snapshot.failures << "\n"
+ << " observations: "
+ << metric_report_snapshot.observations
+ << "\n"
+ << " mean: "
+ << metric_report_snapshot.latency.Mean()
+ << " us\n"
+ << " p50 / p90 / p99: "
+ << metric_report_snapshot.latency.Percentile(50) << " / "
+ << metric_report_snapshot.latency.Percentile(90) << " / "
+ << metric_report_snapshot.latency.Percentile(99) << " us\n"
+ << "\n CFM policy commands executed\n"
+ << " evictions: " << eviction_commands << "\n"
+ << " prefetches: " << prefetch_commands << "\n"
+ << " admissions: " << admission_commands << "\n"
+ << " policy drain: "
+ << (embedded_service
+ ? (policy_drain_complete ? "complete" : "timed out/error")
+ : "remote endpoint is not verifiable")
+ << "\n"
+ << "\n IO Pattern snapshots\n"
+ << " client keys / storage: " << source_snapshot.keys.size() << " / "
+ << source_snapshot.storage.size() << "\n";
+ if (embedded_service) {
+ std::cout << " CFM keys / storage: " << cfm_snapshot.keys.size()
+ << " / " << cfm_snapshot.storage.size() << "\n";
+ } else {
+ std::cout << " CFM keys / storage: remote endpoint (not exposed to "
+ "the client)\n";
+ }
+ PrintObservability("Client", source_metrics);
+ if (embedded_service) PrintObservability("CFM", cfm_metrics);
+ std::cout << "============================================================\n";
+ return failed_reports == 0 ? 0 : 2;
+}
diff --git a/mooncake-store/benchmarks/vchunk_distributed_bench.cpp b/mooncake-store/benchmarks/vchunk_distributed_bench.cpp
new file mode 100644
index 0000000000..712684ce27
--- /dev/null
+++ b/mooncake-store/benchmarks/vchunk_distributed_bench.cpp
@@ -0,0 +1,229 @@
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "master_client.h"
+#include "vchunk_client.h"
+#include "vchunk_control_plane.h"
+#include "vchunk_transfer_engine.h"
+
+DEFINE_string(master, "127.0.0.1:50051", "Master RPC address");
+DEFINE_string(routing_etcd, "",
+ "ETCD endpoints used to load the CVM SubMaster route snapshot");
+DEFINE_string(cluster_namespace, "",
+ "CVM cluster namespace; required with --routing_etcd");
+DEFINE_string(metadata, "http://127.0.0.1:8080/metadata",
+ "TransferEngine metadata connection string");
+DEFINE_string(local_server, "vchunk-bench-client:12345",
+ "Unique TransferEngine client endpoint");
+DEFINE_string(protocol, "tcp", "TransferEngine transport protocol");
+DEFINE_bool(auto_discovery, false,
+ "Enable TransferEngine device auto-discovery (normally required "
+ "for RDMA without an explicit device matrix)");
+DEFINE_string(tenant, "vchunk-benchmark", "Tenant id");
+DEFINE_uint64(object_size, 1048576, "Object size in bytes");
+DEFINE_uint64(operations, 1000, "Put/Get/Remove transactions");
+DEFINE_uint32(concurrency, 8, "Concurrent workers");
+DEFINE_uint32(timeout_ms, 30000, "Per-operation timeout");
+
+namespace mooncake {
+namespace {
+using Clock = std::chrono::steady_clock;
+
+class DisabledLegacyPath final : public VChunkLegacyPath {
+ public:
+ ErrorCode Put(const TenantId&, const std::string&, const void*, size_t)
+ override {
+ return ErrorCode::INVALID_PARAMS;
+ }
+ ErrorCode Get(const TenantId&, const std::string&, void*, size_t) override {
+ return ErrorCode::INVALID_PARAMS;
+ }
+ ErrorCode Remove(const TenantId&, const std::string&) override {
+ return ErrorCode::INVALID_PARAMS;
+ }
+};
+
+double Percentile(std::vector values, double percentile) {
+ if (values.empty()) return 0;
+ std::sort(values.begin(), values.end());
+ const auto index = static_cast(
+ percentile * static_cast(values.size() - 1));
+ return values[index];
+}
+
+ErrorCode ConfigureRouting(MasterClient& master) {
+ if (FLAGS_routing_etcd.empty() && FLAGS_cluster_namespace.empty()) {
+ return ErrorCode::OK;
+ }
+ if (FLAGS_routing_etcd.empty() || FLAGS_cluster_namespace.empty()) {
+ return ErrorCode::INVALID_PARAMS;
+ }
+ return master.LoadRoutingFromEtcd(FLAGS_routing_etcd,
+ FLAGS_cluster_namespace);
+}
+} // namespace
+} // namespace mooncake
+
+int main(int argc, char** argv) {
+ gflags::ParseCommandLineFlags(&argc, &argv, true);
+ google::InitGoogleLogging(argv[0]);
+ using namespace mooncake;
+ if (FLAGS_object_size == 0 || FLAGS_operations == 0 ||
+ FLAGS_concurrency == 0 || FLAGS_timeout_ms == 0) {
+ LOG(ERROR) << "benchmark numeric arguments must be greater than zero";
+ return 2;
+ }
+
+ MasterClient master(generate_uuid(), nullptr, FLAGS_tenant);
+ if (const auto error = master.Connect(FLAGS_master);
+ error != ErrorCode::OK) {
+ LOG(ERROR) << "failed to connect master: " << toString(error);
+ return 2;
+ }
+ if (const auto error = ConfigureRouting(master); error != ErrorCode::OK) {
+ LOG(ERROR) << "failed to load SubMaster routing: " << toString(error);
+ return 2;
+ }
+ auto runtime = master.GetVChunkRuntimeInfo();
+ if (!runtime || !runtime->enabled || !runtime->persistent_metadata) {
+ LOG(ERROR) << "master must enable vchunk with persistent ETCD metadata";
+ return 2;
+ }
+ TransferEngine engine(FLAGS_auto_discovery);
+ if (engine.init(FLAGS_metadata, FLAGS_local_server) != 0 ||
+ engine.installTransport(FLAGS_protocol, nullptr) == nullptr) {
+ LOG(ERROR) << "failed to initialize TransferEngine";
+ return 2;
+ }
+ TransferEngineVChunkDataPlane data_plane(engine);
+ DisabledLegacyPath legacy;
+ const TenantId tenant(FLAGS_tenant);
+
+ std::atomic next{0};
+ std::atomic succeeded{0};
+ std::atomic failed{0};
+ std::vector latencies(FLAGS_operations);
+ std::vector workers;
+ workers.reserve(FLAGS_concurrency);
+ std::atomic setup_failed{false};
+ std::barrier start_barrier(FLAGS_concurrency + 1);
+ std::barrier finish_barrier(FLAGS_concurrency + 1);
+ for (uint32_t worker_id = 0; worker_id < FLAGS_concurrency; ++worker_id) {
+ workers.emplace_back([&, worker_id] {
+ MasterClient worker_master(generate_uuid(), nullptr, FLAGS_tenant);
+ const auto connect_error = worker_master.Connect(FLAGS_master);
+ const auto routing_error =
+ connect_error == ErrorCode::OK
+ ? ConfigureRouting(worker_master)
+ : connect_error;
+ RpcVChunkControlPlane control_plane(worker_master);
+ std::vector source(FLAGS_object_size);
+ std::vector destination(FLAGS_object_size);
+ const bool source_registered =
+ engine.registerLocalMemory(source.data(), source.size(),
+ "cpu:0") == 0;
+ const bool destination_registered =
+ engine.registerLocalMemory(destination.data(),
+ destination.size(), "cpu:0") == 0;
+ if (!source_registered || !destination_registered ||
+ routing_error != ErrorCode::OK) {
+ setup_failed.store(true);
+ }
+ start_barrier.arrive_and_wait();
+ if (setup_failed.load()) {
+ finish_barrier.arrive_and_wait();
+ if (source_registered)
+ engine.unregisterLocalMemory(source.data());
+ if (destination_registered)
+ engine.unregisterLocalMemory(destination.data());
+ return;
+ }
+ VChunkClient client(
+ true, control_plane, data_plane, legacy,
+ std::chrono::milliseconds(FLAGS_timeout_ms),
+ [] { return getCurrentTimeInMilli(); });
+ while (true) {
+ const auto operation = next.fetch_add(1);
+ if (operation >= FLAGS_operations) break;
+ for (size_t i = 0; i < source.size(); ++i) {
+ source[i] = static_cast((operation + i) & 0xff);
+ }
+ const auto key = "distributed-" +
+ std::to_string(worker_id) + "-" +
+ std::to_string(operation);
+ const auto operation_started = Clock::now();
+ auto error = client.Put(tenant, key, source.data(),
+ source.size());
+ if (error == ErrorCode::OK) {
+ error = client.Get(tenant, key, destination.data(),
+ destination.size());
+ }
+ const bool data_matches = destination == source;
+ const auto remove_error = client.Remove(tenant, key);
+ latencies[operation] =
+ std::chrono::duration(
+ Clock::now() - operation_started)
+ .count();
+ if (error == ErrorCode::OK && data_matches &&
+ remove_error == ErrorCode::OK) {
+ succeeded.fetch_add(1);
+ } else {
+ failed.fetch_add(1);
+ }
+ }
+ finish_barrier.arrive_and_wait();
+ engine.unregisterLocalMemory(source.data());
+ engine.unregisterLocalMemory(destination.data());
+ });
+ }
+ start_barrier.arrive_and_wait();
+ const auto started = Clock::now();
+ finish_barrier.arrive_and_wait();
+ const auto finished = Clock::now();
+ for (auto& worker : workers) worker.join();
+ if (setup_failed.load()) {
+ LOG(ERROR) << "failed to register benchmark buffers";
+ return 2;
+ }
+ const auto seconds =
+ std::chrono::duration(finished - started).count();
+ latencies.resize(static_cast(next.load() > FLAGS_operations
+ ? FLAGS_operations
+ : next.load()));
+ const double gib = static_cast(succeeded.load()) *
+ static_cast(FLAGS_object_size) * 2.0 /
+ (1024.0 * 1024.0 * 1024.0);
+ std::cout << "{\n"
+ << " \"production_equivalent_data_plane\": true,\n"
+ << " \"master_rpc\": \"" << FLAGS_master << "\",\n"
+ << " \"submaster_routing\": "
+ << (!FLAGS_routing_etcd.empty() ? "true" : "false") << ",\n"
+ << " \"cluster_namespace\": \"" << FLAGS_cluster_namespace
+ << "\",\n"
+ << " \"transfer_protocol\": \"" << FLAGS_protocol << "\",\n"
+ << " \"auto_discovery\": "
+ << (FLAGS_auto_discovery ? "true" : "false") << ",\n"
+ << " \"object_size_bytes\": " << FLAGS_object_size << ",\n"
+ << " \"concurrency\": " << FLAGS_concurrency << ",\n"
+ << " \"operations\": " << FLAGS_operations << ",\n"
+ << " \"succeeded\": " << succeeded.load() << ",\n"
+ << " \"failed\": " << failed.load() << ",\n"
+ << " \"duration_seconds\": " << seconds << ",\n"
+ << " \"transactions_per_second\": "
+ << succeeded.load() / seconds << ",\n"
+ << " \"data_gib_per_second\": " << gib / seconds << ",\n"
+ << " \"latency_p50_us\": " << Percentile(latencies, 0.50)
+ << ",\n"
+ << " \"latency_p99_us\": " << Percentile(latencies, 0.99)
+ << "\n}\n";
+ return failed.load() == 0 && succeeded.load() == FLAGS_operations ? 0 : 1;
+}
diff --git a/mooncake-store/conf/master.json b/mooncake-store/conf/master.json
index f55bdaf2b2..01757fdc2d 100644
--- a/mooncake-store/conf/master.json
+++ b/mooncake-store/conf/master.json
@@ -15,6 +15,14 @@
"enable_ha": false,
"enable_oplog": false,
"etcd_endpoints": "http://localhost:2379",
+ "enable_vchunk": false,
+ "vchunk_etcd_endpoints": "",
+ "vchunk_creating_timeout_ms": 30000,
+ "vchunk_releasing_timeout_ms": 60000,
+ "vchunk_max_slice_retry": 3,
+ "vchunk_max_slice_count": 4096,
+ "vchunk_max_metadata_bytes": 1048576,
+ "vchunk_max_creating_objects": 1024,
"root_fs_dir": "",
"cluster_id": "mooncake_cluster",
"memory_allocator": "offset",
diff --git a/mooncake-store/conf/master.yaml b/mooncake-store/conf/master.yaml
index 545807008f..7f9bfd7f94 100644
--- a/mooncake-store/conf/master.yaml
+++ b/mooncake-store/conf/master.yaml
@@ -21,6 +21,16 @@ tenant_quota_connector_uri: ""
enable_ha: false
etcd_endpoints: "http://localhost:2379"
+# Distributed vchunk validation path. Keep disabled for the community path.
+enable_vchunk: false
+# Empty means reuse etcd_endpoints.
+vchunk_etcd_endpoints: ""
+vchunk_creating_timeout_ms: 30000
+vchunk_releasing_timeout_ms: 60000
+vchunk_max_slice_retry: 3
+vchunk_max_slice_count: 4096
+vchunk_max_metadata_bytes: 1048576
+vchunk_max_creating_objects: 1024
root_fs_dir: ""
cluster_id: "mooncake_cluster"
memory_allocator: "offset"
diff --git a/mooncake-store/graphify-out/cache/stat-index.json b/mooncake-store/graphify-out/cache/stat-index.json
new file mode 100644
index 0000000000..26f3fefc57
--- /dev/null
+++ b/mooncake-store/graphify-out/cache/stat-index.json
@@ -0,0 +1 @@
+{"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\AGENTS.md":{"size":629,"mtime_ns":1786963737729753600,"word_count":89},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\CMakeLists.txt":{"size":3012,"mtime_ns":1786963737730754700,"word_count":175},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\CMakeLists.txt":{"size":3960,"mtime_ns":1788421144221545100,"word_count":247},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\README.md":{"size":3145,"mtime_ns":1786963737730754700,"word_count":367},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\allocation_strategy_bench.cpp":{"size":79226,"mtime_ns":1786963737731754800,"word_count":7249},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\allocator_bench.cpp":{"size":11232,"mtime_ns":1786963737731754800,"word_count":1091},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\batch_evict_bench.cpp":{"size":18064,"mtime_ns":1786963737732754700,"word_count":1293},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\batch_get_replica_bench.cpp":{"size":28446,"mtime_ns":1786963737732754700,"word_count":2261},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\batch_remove_benchmark.py":{"size":14812,"mtime_ns":1781678985149681300,"word_count":1186},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\cluster_mooncake_diag.py":{"size":97859,"mtime_ns":1786976645759111200,"word_count":8162},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\file_interface_bench.cpp":{"size":34867,"mtime_ns":1781678985149681300,"word_count":3178},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\master_bench.cpp":{"size":19966,"mtime_ns":1786963737733754900,"word_count":1578},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\nof_worker_pool_bench.cpp":{"size":40711,"mtime_ns":1786976645759111200,"word_count":3092},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\oplog_batch_bench.cpp":{"size":13726,"mtime_ns":1786963737733754900,"word_count":993},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\report_oplog_batch.py":{"size":7146,"mtime_ns":1786963737733754900,"word_count":512},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\run_oplog_batch_sweep.py":{"size":7127,"mtime_ns":1786963737733754900,"word_count":475},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\storage_backend_bench.cpp":{"size":94331,"mtime_ns":1781678985150683700,"word_count":8403},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\store_connector_bench.cpp":{"size":19331,"mtime_ns":1788421144221545100,"word_count":1730},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\store_kv_bench.md":{"size":5588,"mtime_ns":1781678985150683700,"word_count":695},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\store_kv_bench.py":{"size":58260,"mtime_ns":1786963737735256300,"word_count":4164},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\stress_cluster_bench.cpp":{"size":74873,"mtime_ns":1788421144221545100,"word_count":6667},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\stress_cluster_ranges_bench.cpp":{"size":42576,"mtime_ns":1786976645760111100,"word_count":4052},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\test_report_oplog_batch.py":{"size":3598,"mtime_ns":1786963737736257800,"word_count":197},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\test_run_oplog_batch_sweep.py":{"size":4288,"mtime_ns":1786963737736257800,"word_count":259},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\test_store_kv_bench.py":{"size":6253,"mtime_ns":1786963737737258000,"word_count":404},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\benchmarks\\vchunk_distributed_bench.cpp":{"size":10326,"mtime_ns":1788421144221545100,"word_count":800},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\conf\\master.json":{"size":1323,"mtime_ns":1788421144221545100,"word_count":80},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\conf\\master.yaml":{"size":1516,"mtime_ns":1788421144221545100,"word_count":132},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\build.sh":{"size":2932,"mtime_ns":1786963737737258000,"word_count":309},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\examples\\basic\\main.go":{"size":2404,"mtime_ns":1781678985152689000,"word_count":318},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\go.mod":{"size":68,"mtime_ns":1781678985152689000,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\mooncakestore\\config.go":{"size":1066,"mtime_ns":1781678985153691400,"word_count":141},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\mooncakestore\\errors.go":{"size":1650,"mtime_ns":1786963737738730400,"word_count":180},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\mooncakestore\\store.go":{"size":14159,"mtime_ns":1781678985153691400,"word_count":1791},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\go\\tests\\integration_test.go":{"size":4372,"mtime_ns":1781678985153691400,"word_count":605},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\admission_ops.h":{"size":45,"mtime_ns":1788421144377666800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\aligned_client_buffer.h":{"size":2139,"mtime_ns":1786963737738730400,"word_count":257},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\allocation_strategy.h":{"size":31158,"mtime_ns":1786963737738730400,"word_count":2636},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\allocator.h":{"size":12151,"mtime_ns":1786976645760111100,"word_count":1075},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\AllocationClass.h":{"size":20014,"mtime_ns":1786963737739832900,"word_count":2544},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\MemoryAllocator.h":{"size":20451,"mtime_ns":1786963737739832900,"word_count":2689},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\MemoryPool.h":{"size":15290,"mtime_ns":1786963737740834200,"word_count":2098},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\MemoryPoolManager.h":{"size":5543,"mtime_ns":1781678985155735100,"word_count":791},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\Slab.h":{"size":10734,"mtime_ns":1781678985155735100,"word_count":1341},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\SlabAllocator.h":{"size":8189,"mtime_ns":1781678985156737400,"word_count":1056},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\common\\CompilerUtils.h":{"size":2349,"mtime_ns":1781678985156737400,"word_count":286},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\common\\Exceptions.h":{"size":3014,"mtime_ns":1781678985156737400,"word_count":334},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\common\\Throttler.h":{"size":3643,"mtime_ns":1781678985156737400,"word_count":459},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\common\\Time.h":{"size":3123,"mtime_ns":1781678985156737400,"word_count":346},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\common\\Utils.h":{"size":1068,"mtime_ns":1781678985157739700,"word_count":165},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\fake_include\\folly\\logging\\xlog.h":{"size":30964,"mtime_ns":1781678985157739700,"word_count":3245},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\fake_include\\folly\\portability\\Config.h":{"size":1116,"mtime_ns":1781678985157739700,"word_count":145},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\args.h":{"size":7699,"mtime_ns":1781678985158742500,"word_count":716},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\chrono.h":{"size":43744,"mtime_ns":1781678985158742500,"word_count":4420},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\color.h":{"size":24752,"mtime_ns":1781678985158742500,"word_count":2221},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\compile.h":{"size":22149,"mtime_ns":1781678985159744700,"word_count":2203},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\core.h":{"size":101361,"mtime_ns":1781678985159744700,"word_count":10856},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\format-inl.h":{"size":106523,"mtime_ns":1781678985160746500,"word_count":9714},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\format.h":{"size":106448,"mtime_ns":1781678985160746500,"word_count":11421},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\locale.h":{"size":102,"mtime_ns":1781678985160746500,"word_count":11},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\os.h":{"size":15582,"mtime_ns":1781678985161748200,"word_count":1809},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\ostream.h":{"size":6152,"mtime_ns":1781678985161748200,"word_count":573},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\printf.h":{"size":21091,"mtime_ns":1781678985161748200,"word_count":2222},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\ranges.h":{"size":14927,"mtime_ns":1781678985161748200,"word_count":1478},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\fmt\\xchar.h":{"size":9467,"mtime_ns":1781678985162750600,"word_count":794},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\folly\\CPortability.h":{"size":10773,"mtime_ns":1781678985162750600,"word_count":1051},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\folly\\Likely.h":{"size":2195,"mtime_ns":1781678985162750600,"word_count":314},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\folly\\Portability.h":{"size":16275,"mtime_ns":1781678985162750600,"word_count":1906},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cachelib_memory_allocator\\include\\folly\\lang\\Builtin.h":{"size":2198,"mtime_ns":1781678985162750600,"word_count":237},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cfm_client.h":{"size":48,"mtime_ns":1788421144377666800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cfm_client_impl.h":{"size":55,"mtime_ns":1788421144492018000,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\client_buffer.h":{"size":5113,"mtime_ns":1786963737740834200,"word_count":552},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\client_metric.h":{"size":28657,"mtime_ns":1786976645761558400,"word_count":2319},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\client_service.h":{"size":44657,"mtime_ns":1788421144221545100,"word_count":4178},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\collector_impl.h":{"size":54,"mtime_ns":1788421144492018000,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\config_helper.h":{"size":2756,"mtime_ns":1781678985164755800,"word_count":352},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\count_min_sketch.h":{"size":2847,"mtime_ns":1781678985164755800,"word_count":332},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\crc32c.h":{"size":1348,"mtime_ns":1786963737741834400,"word_count":184},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\cvm_controller.h":{"size":5947,"mtime_ns":1788421144227958400,"word_count":485},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\cvm_http_server.h":{"size":1763,"mtime_ns":1788421144227958400,"word_count":132},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\cvm_keys.h":{"size":3551,"mtime_ns":1788421144227958400,"word_count":298},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\cvm_service_delegate.h":{"size":1329,"mtime_ns":1788421144227958400,"word_count":177},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\cvm_types.h":{"size":3902,"mtime_ns":1788421144229244100,"word_count":369},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\etcd_view_store.h":{"size":7208,"mtime_ns":1788421144229244100,"word_count":546},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\inter_master_rpc.h":{"size":8136,"mtime_ns":1788421144229244100,"word_count":867},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\slot_hash.h":{"size":6206,"mtime_ns":1788421144230337600,"word_count":674},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\slot_migrator.h":{"size":2634,"mtime_ns":1788421144230337600,"word_count":299},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\cvm\\slot_owner_heartbeat.h":{"size":3600,"mtime_ns":1788421144230337600,"word_count":408},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\deadline_scheduler.h":{"size":4366,"mtime_ns":1786963737741834400,"word_count":306},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\degrading_policy_engine.h":{"size":63,"mtime_ns":1788421144492018000,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\device\\accelerator_device.h":{"size":1495,"mtime_ns":1786963737742939400,"word_count":159},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\device\\accelerator_registry.h":{"size":796,"mtime_ns":1786963737742939400,"word_count":72},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\device\\runtime_accelerator.h":{"size":772,"mtime_ns":1786963737742939400,"word_count":74},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\dummy_client.h":{"size":11336,"mtime_ns":1786963737742939400,"word_count":900},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\engram\\engram_store.h":{"size":2660,"mtime_ns":1781678985164755800,"word_count":309},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\engram\\engram_store_config.h":{"size":552,"mtime_ns":1781678985164755800,"word_count":76},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\etcd_helper.h":{"size":12275,"mtime_ns":1788421144230337600,"word_count":1472},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\eviction_ops.h":{"size":45,"mtime_ns":1788421144377666800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\eviction_strategy.h":{"size":3155,"mtime_ns":1781678985165755600,"word_count":278},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\feedback.h":{"size":48,"mtime_ns":1788421144492018000,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\file_interface.h":{"size":9736,"mtime_ns":1786963737744372300,"word_count":1001},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\file_storage.h":{"size":6514,"mtime_ns":1788421144230337600,"word_count":596},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\common\\redis\\redis_connection.h":{"size":1321,"mtime_ns":1781678985166263700,"word_count":112},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\ha_types.h":{"size":5851,"mtime_ns":1788421144231831000,"word_count":493},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\kv\\etcd_ha_kv_backend.h":{"size":545,"mtime_ns":1786963737745374500,"word_count":50},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\kv\\ha_kv_backend.h":{"size":1031,"mtime_ns":1786963737745374500,"word_count":100},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\backends\\etcd\\etcd_leader_coordinator.h":{"size":2693,"mtime_ns":1786963737746760700,"word_count":177},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\backends\\k8s\\k8s_leader_coordinator.h":{"size":2345,"mtime_ns":1781678985167265600,"word_count":155},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\backends\\redis\\redis_leader_coordinator.h":{"size":3305,"mtime_ns":1781678985167265600,"word_count":210},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\leader_coordinator.h":{"size":1284,"mtime_ns":1781678985167265600,"word_count":101},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\leader_coordinator_factory.h":{"size":353,"mtime_ns":1781678985168265800,"word_count":29},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\leader_label_reconciler.h":{"size":2997,"mtime_ns":1786963737746760700,"word_count":291},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\leadership\\master_service_supervisor.h":{"size":356,"mtime_ns":1781678985168265800,"word_count":33},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\master_metrics_reporter.h":{"size":2974,"mtime_ns":1786976645762816900,"word_count":351},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_applier.h":{"size":2888,"mtime_ns":1786976645764055800,"word_count":312},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_batch_codec.h":{"size":539,"mtime_ns":1786963737747919400,"word_count":41},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_batch_standby_reader.h":{"size":1226,"mtime_ns":1788421144231831000,"word_count":81},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_batch_storage.h":{"size":1301,"mtime_ns":1788421144231831000,"word_count":104},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_batch_types.h":{"size":2033,"mtime_ns":1788421144231831000,"word_count":168},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_test_failpoint.h":{"size":179,"mtime_ns":1786963737747919400,"word_count":20},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\oplog_types.h":{"size":1820,"mtime_ns":1786976645764055800,"word_count":150},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\oplog\\ordered_oplog_writer.h":{"size":2033,"mtime_ns":1786963737749323800,"word_count":135},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\catalog\\backends\\embedded\\embedded_snapshot_catalog_store.h":{"size":1058,"mtime_ns":1781678985170791500,"word_count":81},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\catalog\\backends\\redis\\redis_snapshot_catalog_store.h":{"size":1545,"mtime_ns":1781678985170791500,"word_count":110},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\catalog\\snapshot_catalog_store.h":{"size":5587,"mtime_ns":1786976645764055800,"word_count":475},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\catalog_backed_snapshot_provider.h":{"size":584,"mtime_ns":1781678985171793800,"word_count":58},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\master_snapshot_codec.h":{"size":5886,"mtime_ns":1786976645765517900,"word_count":583},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\object\\backends\\local\\local_file_snapshot_object_store.h":{"size":1937,"mtime_ns":1781678985171793800,"word_count":157},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\object\\backends\\s3\\s3_snapshot_object_store.h":{"size":1538,"mtime_ns":1781678985171793800,"word_count":123},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\object\\snapshot_object_store.h":{"size":4680,"mtime_ns":1781678985171793800,"word_count":532},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\snapshot_constants.h":{"size":916,"mtime_ns":1786976645765517900,"word_count":93},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\snapshot_logger.h":{"size":1226,"mtime_ns":1781678985171793800,"word_count":128},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\snapshot\\snapshot_provider.h":{"size":1788,"mtime_ns":1786976645765517900,"word_count":193},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha\\standby_controller.h":{"size":1641,"mtime_ns":1788421144233316700,"word_count":157},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ha_metric_manager.h":{"size":8826,"mtime_ns":1786963737750325300,"word_count":707},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\hf3fs\\hf3fs.h":{"size":3460,"mtime_ns":1781678985172795700,"word_count":342},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\hot_standby_service.h":{"size":12085,"mtime_ns":1788421144233316700,"word_count":1188},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\http_metadata_server.h":{"size":1693,"mtime_ns":1786963737751598000,"word_count":179},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\hybrid_metric.h":{"size":15710,"mtime_ns":1781678985173797700,"word_count":1131},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern.h":{"size":52,"mtime_ns":1788421144378944000,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\analyzer.h":{"size":591,"mtime_ns":1788421144378944000,"word_count":59},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\cfm_channel.h":{"size":1598,"mtime_ns":1788421144627563800,"word_count":147},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\cfm_client_impl.h":{"size":1121,"mtime_ns":1788421144493421600,"word_count":88},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\cfm_ingress.h":{"size":872,"mtime_ns":1788421144627563800,"word_count":67},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\cfm_protocol.h":{"size":923,"mtime_ns":1788421144493421600,"word_count":84},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\cfm_service.h":{"size":4393,"mtime_ns":1788426179828211900,"word_count":350},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\client.h":{"size":520,"mtime_ns":1788421144378944000,"word_count":56},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\collector.h":{"size":692,"mtime_ns":1788421144493421600,"word_count":70},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\collector_impl.h":{"size":3003,"mtime_ns":1788440556672660100,"word_count":229},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\degrading_policy_engine.h":{"size":1509,"mtime_ns":1788421144494781100,"word_count":113},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\feedback.h":{"size":1781,"mtime_ns":1788421144494781100,"word_count":144},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\io_pattern.h":{"size":1060,"mtime_ns":1788421144634371100,"word_count":56},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\kmeans_analyzer.h":{"size":938,"mtime_ns":1788440380997173000,"word_count":84},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\legacy_eviction_ops.h":{"size":635,"mtime_ns":1788421144494781100,"word_count":48},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\observability.h":{"size":1169,"mtime_ns":1788421144496090400,"word_count":84},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\ops.h":{"size":1622,"mtime_ns":1788421144496333600,"word_count":179},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\policy_engine.h":{"size":15132,"mtime_ns":1788421144496333600,"word_count":979},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\policy_strategies.h":{"size":2102,"mtime_ns":1788421144497337700,"word_count":155},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\registry.h":{"size":1850,"mtime_ns":1788421144380824800,"word_count":143},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\reporter.h":{"size":2065,"mtime_ns":1788421144634371100,"word_count":163},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\resilient_analyzer.h":{"size":1397,"mtime_ns":1788421144497337700,"word_count":118},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\resilient_cfm_channel.h":{"size":1256,"mtime_ns":1788421144635449700,"word_count":103},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\rpc_transport.h":{"size":6678,"mtime_ns":1788421144635449700,"word_count":559},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\runtime.h":{"size":5344,"mtime_ns":1788440557542777000,"word_count":394},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\sliding_window_analyzer.h":{"size":1837,"mtime_ns":1788421144636607800,"word_count":134},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\threshold_analyzer.h":{"size":1368,"mtime_ns":1788421144498337300,"word_count":108},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\tier_executor.h":{"size":1215,"mtime_ns":1788440868041497900,"word_count":86},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern\\types.h":{"size":6727,"mtime_ns":1788421144499337400,"word_count":516},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern_analyzer.h":{"size":94,"mtime_ns":1788421144499337400,"word_count":6},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern_collector.h":{"size":51,"mtime_ns":1788421144380824800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern_registry.h":{"size":50,"mtime_ns":1788421144382219800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\io_pattern_types.h":{"size":47,"mtime_ns":1788421144382219800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\k8s_lease_helper.h":{"size":1837,"mtime_ns":1786963737752099900,"word_count":139},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\kv_event\\key_util.h":{"size":992,"mtime_ns":1786963737752099900,"word_count":105},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\kv_event\\kv_event_config.h":{"size":1357,"mtime_ns":1786963737752099900,"word_count":149},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\kv_event\\kv_event_publisher.h":{"size":3873,"mtime_ns":1786963737753101300,"word_count":298},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\legacy_eviction_ops.h":{"size":59,"mtime_ns":1788421144499337400,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\local_hot_cache.h":{"size":11371,"mtime_ns":1786160474197952500,"word_count":1370},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_admin_service.h":{"size":4897,"mtime_ns":1786963737753101300,"word_count":241},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_client.h":{"size":39042,"mtime_ns":1788421144234318200,"word_count":3659},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_config.h":{"size":72889,"mtime_ns":1788439904229877000,"word_count":4717},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_metric_manager.h":{"size":33987,"mtime_ns":1786963737754101400,"word_count":2293},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_perf.h":{"size":508,"mtime_ns":1786976645768275400,"word_count":61},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_service.h":{"size":119181,"mtime_ns":1788421144637865000,"word_count":10005},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_snapshot_manager.h":{"size":4247,"mtime_ns":1786963737755103100,"word_count":298},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\master_snapshot_repository.h":{"size":4236,"mtime_ns":1786963737756103100,"word_count":425},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\memory_alloc.h":{"size":140,"mtime_ns":1781678985175819100,"word_count":12},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\metadata_store.h":{"size":7441,"mtime_ns":1788421144236318000,"word_count":755},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\mmap_arena.h":{"size":3516,"mtime_ns":1781678985175819100,"word_count":391},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\mutex.h":{"size":8930,"mtime_ns":1781678985175819100,"word_count":873},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\observability.h":{"size":53,"mtime_ns":1788421144500336600,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\offset_allocator\\offset_allocator.h":{"size":17218,"mtime_ns":1786963737756103100,"word_count":1553},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\partition\\kv_hash_map.h":{"size":603,"mtime_ns":1788421144236318000,"word_count":62},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\partition\\partition_router.h":{"size":1346,"mtime_ns":1788421144236318000,"word_count":105},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\pinned_buffer_pool.h":{"size":4319,"mtime_ns":1786963737757223600,"word_count":372},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\pinned_host_buffer.h":{"size":1376,"mtime_ns":1786963737757223600,"word_count":131},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\policy_engine.h":{"size":55,"mtime_ns":1788421144382219800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\policy_strategies.h":{"size":59,"mtime_ns":1788421144500336600,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\prefetch_ops.h":{"size":45,"mtime_ns":1788421144382219800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\pyclient.h":{"size":19997,"mtime_ns":1786976645769276700,"word_count":1655},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\random.h":{"size":2660,"mtime_ns":1786963737757223600,"word_count":231},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\real_client.h":{"size":41321,"mtime_ns":1788009798331033800,"word_count":3486},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\replica.h":{"size":25893,"mtime_ns":1786976645770732100,"word_count":2030},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\replica_selection.h":{"size":6874,"mtime_ns":1786963737759606600,"word_count":806},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\reporter.h":{"size":48,"mtime_ns":1788421144500336600,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\resilient_analyzer.h":{"size":58,"mtime_ns":1788421144501826800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\resilient_cfm_channel.h":{"size":61,"mtime_ns":1788421144501826800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\rpc_helper.h":{"size":4652,"mtime_ns":1786976645770732100,"word_count":462},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\rpc_service.h":{"size":19067,"mtime_ns":1788421144637865000,"word_count":1350},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\rpc_transport.h":{"size":53,"mtime_ns":1788421144501826800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\rpc_types.h":{"size":11159,"mtime_ns":1788421144237318100,"word_count":851},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\segment.h":{"size":22925,"mtime_ns":1788421144237318100,"word_count":1999},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\serialize\\serializer.h":{"size":4779,"mtime_ns":1786963737760606500,"word_count":359},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\serializer.h":{"size":12547,"mtime_ns":1781678985178825800,"word_count":1559},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\shm_helper.h":{"size":1423,"mtime_ns":1786963737761968200,"word_count":159},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\sliding_window_analyzer.h":{"size":63,"mtime_ns":1788421144501826800,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\spdk\\spdk_wrapper.h":{"size":3550,"mtime_ns":1781678985179829300,"word_count":258},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\ssd_register_client.h":{"size":1232,"mtime_ns":1781678985179829300,"word_count":115},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\standby_state_machine.h":{"size":11769,"mtime_ns":1781678985179829300,"word_count":971},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\storage\\distributed\\distributed_storage_backend.h":{"size":2253,"mtime_ns":1786963737761968200,"word_count":164},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\storage\\distributed\\fs_adapter.h":{"size":3420,"mtime_ns":1781678985180832500,"word_count":334},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\storage\\distributed\\hf3fs_adapter.h":{"size":1868,"mtime_ns":1781678985180832500,"word_count":141},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\storage_backend.h":{"size":72796,"mtime_ns":1788421144238318300,"word_count":7544},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\store_c.h":{"size":5260,"mtime_ns":1781678985180832500,"word_count":410},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\store_rpc_client_io_context.h":{"size":568,"mtime_ns":1786963737762969600,"word_count":43},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\task_manager.h":{"size":7883,"mtime_ns":1781678985181835400,"word_count":627},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\tenant_id.h":{"size":2667,"mtime_ns":1786963737762969600,"word_count":241},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\tenant_quota.h":{"size":4201,"mtime_ns":1786963737762969600,"word_count":323},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\tenant_quota_policy_store.h":{"size":1984,"mtime_ns":1786963737764195500,"word_count":142},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\tenant_quota_sharded.h":{"size":2772,"mtime_ns":1786963737764195500,"word_count":200},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\tenant_quota_sharded_impl.h":{"size":9132,"mtime_ns":1786963737764195500,"word_count":581},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\thread_pool.h":{"size":2271,"mtime_ns":1781678985181835400,"word_count":264},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\threshold_analyzer.h":{"size":60,"mtime_ns":1788421144502828700,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\tier_executor.h":{"size":53,"mtime_ns":1788421144502828700,"word_count":4},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\transfer_task.h":{"size":21678,"mtime_ns":1786976645774403300,"word_count":1939},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\types.h":{"size":21758,"mtime_ns":1788421144238318300,"word_count":2444},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\uds_transport.h":{"size":2240,"mtime_ns":1786963737765196800,"word_count":203},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils.h":{"size":17970,"mtime_ns":1786976645774403300,"word_count":2213},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils\\base64.h":{"size":4714,"mtime_ns":1781678985182835000,"word_count":629},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils\\file_util.h":{"size":1263,"mtime_ns":1781678985182835000,"word_count":145},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils\\s3_helper.h":{"size":2664,"mtime_ns":1781678985183344100,"word_count":204},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils\\scoped_vlog_timer.h":{"size":4435,"mtime_ns":1781678985183344100,"word_count":390},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils\\type_util.h":{"size":1255,"mtime_ns":1786976645775882200,"word_count":166},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\utils\\zstd_util.h":{"size":8044,"mtime_ns":1781678985183344100,"word_count":564},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_allocation_strategy.h":{"size":1567,"mtime_ns":1788421144239424500,"word_count":111},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_client.h":{"size":3675,"mtime_ns":1788421144239424500,"word_count":271},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_config.h":{"size":1151,"mtime_ns":1788421144239424500,"word_count":90},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_control_plane.h":{"size":3011,"mtime_ns":1788421144239424500,"word_count":255},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_master_manager.h":{"size":3572,"mtime_ns":1788421144239424500,"word_count":267},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_metadata.h":{"size":3019,"mtime_ns":1788421144240919400,"word_count":231},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_metadata_store.h":{"size":2176,"mtime_ns":1788421144240919400,"word_count":192},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_metrics.h":{"size":2270,"mtime_ns":1788421144240919400,"word_count":173},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\include\\vchunk_transfer_engine.h":{"size":1423,"mtime_ns":1788421144240919400,"word_count":92},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\CMakeLists.txt":{"size":2712,"mtime_ns":1786963737766493000,"word_count":189},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\README.md":{"size":4434,"mtime_ns":1786963737767565400,"word_count":494},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\build.rs":{"size":15856,"mtime_ns":1786963737767565400,"word_count":1197},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\examples\\basic_usage.rs":{"size":5645,"mtime_ns":1786963737767565400,"word_count":643},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\examples\\generate_dlopen_bindings.rs":{"size":2078,"mtime_ns":1786963737768775400,"word_count":216},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\examples\\store_benchmark.rs":{"size":4657,"mtime_ns":1781678985185350300,"word_count":385},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\src\\error.rs":{"size":2793,"mtime_ns":1786963737768775400,"word_count":370},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\src\\ffi_dlopen.rs":{"size":9934,"mtime_ns":1786963737768775400,"word_count":1053},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\src\\generated\\ffi_dlopen_bindings.rs":{"size":15653,"mtime_ns":1786963737768775400,"word_count":1065},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\src\\lib.rs":{"size":2322,"mtime_ns":1786963737770142200,"word_count":307},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\src\\store.rs":{"size":27624,"mtime_ns":1786976645775882200,"word_count":2578},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\rust\\tests\\minimal_smoke.rs":{"size":2619,"mtime_ns":1781678985186368700,"word_count":230},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\CMakeLists.txt":{"size":19816,"mtime_ns":1788421144639234500,"word_count":1256},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\aligned_client_buffer.cpp":{"size":6426,"mtime_ns":1786963737771143400,"word_count":544},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\allocator.cpp":{"size":24154,"mtime_ns":1786976645776883700,"word_count":1802},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\AllocationClass.cpp":{"size":24528,"mtime_ns":1786963737771143400,"word_count":2404},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\CMakeLists.txt":{"size":479,"mtime_ns":1781678985187370600,"word_count":17},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\MemoryAllocator.cpp":{"size":10858,"mtime_ns":1786963737772491500,"word_count":1062},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\MemoryPool.cpp":{"size":13939,"mtime_ns":1786963737772992800,"word_count":1481},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\MemoryPoolManager.cpp":{"size":4726,"mtime_ns":1781678985188372500,"word_count":514},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\Slab.cpp":{"size":932,"mtime_ns":1781678985188372500,"word_count":127},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cachelib_memory_allocator\\SlabAllocator.cpp":{"size":5166,"mtime_ns":1781678985188372500,"word_count":520},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\client_buffer.cpp":{"size":6712,"mtime_ns":1786963737772992800,"word_count":527},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\client_metric.cpp":{"size":8798,"mtime_ns":1786976645776883700,"word_count":718},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\client_service.cpp":{"size":209778,"mtime_ns":1788421144242920700,"word_count":16465},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cvm\\cvm_controller.cpp":{"size":21148,"mtime_ns":1788421144242920700,"word_count":1545},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cvm\\cvm_http_server.cpp":{"size":4097,"mtime_ns":1788421144242920700,"word_count":303},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cvm\\etcd_view_store.cpp":{"size":23996,"mtime_ns":1788421144244220900,"word_count":1874},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cvm\\inter_master_rpc.cpp":{"size":16952,"mtime_ns":1788421144244220900,"word_count":1252},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cvm\\slot_migrator.cpp":{"size":11634,"mtime_ns":1788421144244220900,"word_count":849},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\cvm\\slot_owner_heartbeat.cpp":{"size":3042,"mtime_ns":1788421144245222700,"word_count":255},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\accelerator_device.cpp":{"size":696,"mtime_ns":1786963737772992800,"word_count":77},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\accelerator_registry.cpp":{"size":3672,"mtime_ns":1786963737772992800,"word_count":262},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\ascend_accelerator_device.cpp":{"size":3610,"mtime_ns":1786963737772992800,"word_count":287},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\cuda_like_accelerator_device.cpp":{"size":3591,"mtime_ns":1786963737772992800,"word_count":274},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\hip_accelerator_device.cpp":{"size":2709,"mtime_ns":1786963737772992800,"word_count":229},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\runtime_accelerator.cpp":{"size":1851,"mtime_ns":1786963737772992800,"word_count":145},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\device\\sunrise_accelerator_device.cpp":{"size":5723,"mtime_ns":1786963737772992800,"word_count":384},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\dummy_client.cpp":{"size":56320,"mtime_ns":1786963737772992800,"word_count":4231},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\engram\\engram_store.cpp":{"size":12078,"mtime_ns":1781678985190376400,"word_count":1037},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\etcd_helper.cpp":{"size":29363,"mtime_ns":1788421144245222700,"word_count":2212},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\file_storage.cpp":{"size":62632,"mtime_ns":1788421144246222200,"word_count":4961},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\common\\redis\\redis_connection.cpp":{"size":7247,"mtime_ns":1781678985200926800,"word_count":575},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\kv\\etcd_ha_kv_backend.cpp":{"size":2984,"mtime_ns":1786963737772992800,"word_count":225},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\leadership\\backends\\etcd\\etcd_leader_coordinator.cpp":{"size":25917,"mtime_ns":1786976645779883500,"word_count":2261},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\leadership\\backends\\k8s\\k8s_leader_coordinator.cpp":{"size":15668,"mtime_ns":1786976645779883500,"word_count":1079},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\leadership\\backends\\redis\\redis_leader_coordinator.cpp":{"size":31018,"mtime_ns":1786976645780883600,"word_count":2119},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\leadership\\leader_coordinator_factory.cpp":{"size":2006,"mtime_ns":1781678985202984100,"word_count":114},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\leadership\\master_service_supervisor.cpp":{"size":24871,"mtime_ns":1788421144246222200,"word_count":1799},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\master_metrics_reporter.cpp":{"size":7444,"mtime_ns":1786976645780883600,"word_count":670},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_applier.cpp":{"size":10421,"mtime_ns":1788008058567345700,"word_count":838},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_batch_codec.cpp":{"size":9183,"mtime_ns":1786963737772992800,"word_count":748},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_batch_standby_reader.cpp":{"size":5475,"mtime_ns":1788421144247222200,"word_count":360},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_batch_storage.cpp":{"size":12985,"mtime_ns":1788421144247222200,"word_count":1154},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_batch_types.cpp":{"size":5538,"mtime_ns":1788421144247222200,"word_count":509},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_test_failpoint.cpp":{"size":3135,"mtime_ns":1786963737772992800,"word_count":326},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\oplog_types.cpp":{"size":1541,"mtime_ns":1786963737772992800,"word_count":130},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\oplog\\ordered_oplog_writer.cpp":{"size":14881,"mtime_ns":1786976645782157400,"word_count":796},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\catalog\\backends\\embedded\\embedded_snapshot_catalog_store.cpp":{"size":8687,"mtime_ns":1786976645783405900,"word_count":556},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\catalog\\backends\\redis\\redis_snapshot_catalog_store.cpp":{"size":12307,"mtime_ns":1786976645783405900,"word_count":835},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\catalog_backed_snapshot_provider.cpp":{"size":25268,"mtime_ns":1786976645784408300,"word_count":1871},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\master_snapshot_codec.cpp":{"size":6322,"mtime_ns":1786963737772992800,"word_count":489},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\object\\backends\\local\\local_file_snapshot_object_store.cpp":{"size":11665,"mtime_ns":1781678985207015300,"word_count":943},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\object\\backends\\s3\\s3_snapshot_object_store.cpp":{"size":2986,"mtime_ns":1781678985207015300,"word_count":204},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\snapshot\\object\\snapshot_object_store.cpp":{"size":1061,"mtime_ns":1781678985207015300,"word_count":72},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha\\standby_controller.cpp":{"size":13605,"mtime_ns":1788421144248222200,"word_count":816},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ha_metric_manager.cpp":{"size":17497,"mtime_ns":1786963737772992800,"word_count":971},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\hf3fs\\CMakeLists.txt":{"size":214,"mtime_ns":1781678985208017900,"word_count":8},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\hf3fs\\README.md":{"size":1245,"mtime_ns":1781678985208017900,"word_count":147},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\hf3fs\\hf3fs_file.cpp":{"size":11781,"mtime_ns":1781678985208017900,"word_count":1014},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\hf3fs\\hf3fs_resource_manager.cpp":{"size":2635,"mtime_ns":1781678985208017900,"word_count":212},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\hot_standby_service.cpp":{"size":36915,"mtime_ns":1788421144248222200,"word_count":2792},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\http_metadata_server.cpp":{"size":5754,"mtime_ns":1786963737772992800,"word_count":412},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\cfm_client_impl.cpp":{"size":1598,"mtime_ns":1788421144639234500,"word_count":107},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\cfm_ingress.cpp":{"size":2449,"mtime_ns":1788421144640236400,"word_count":207},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\cfm_protocol.cpp":{"size":17532,"mtime_ns":1788421144504174300,"word_count":1327},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\cfm_service.cpp":{"size":13096,"mtime_ns":1788441063931337600,"word_count":933},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\collector_impl.cpp":{"size":11109,"mtime_ns":1788421144640236400,"word_count":755},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\degrading_policy_engine.cpp":{"size":1992,"mtime_ns":1788421144505176000,"word_count":162},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\feedback.cpp":{"size":1962,"mtime_ns":1788421144505176000,"word_count":162},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\kmeans_analyzer.cpp":{"size":6580,"mtime_ns":1788421144505176000,"word_count":533},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\legacy_eviction_ops.cpp":{"size":1468,"mtime_ns":1788421144505176000,"word_count":108},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\observability.cpp":{"size":2154,"mtime_ns":1788421144506176100,"word_count":128},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\policy_strategies.cpp":{"size":9558,"mtime_ns":1788421144641310700,"word_count":705},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\reporter.cpp":{"size":4519,"mtime_ns":1788421144641310700,"word_count":322},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\resilient_analyzer.cpp":{"size":1904,"mtime_ns":1788421144506176100,"word_count":164},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\resilient_cfm_channel.cpp":{"size":2633,"mtime_ns":1788421144641310700,"word_count":205},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\rpc_transport.cpp":{"size":11308,"mtime_ns":1788421144641310700,"word_count":784},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\runtime.cpp":{"size":15196,"mtime_ns":1788440558445802800,"word_count":980},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\sliding_window_analyzer.cpp":{"size":5112,"mtime_ns":1788421144642804600,"word_count":387},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\threshold_analyzer.cpp":{"size":5870,"mtime_ns":1788421144508611900,"word_count":435},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\io_pattern\\tier_executor.cpp":{"size":1423,"mtime_ns":1788421144508611900,"word_count":101},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\k8s_lease_helper.cpp":{"size":9412,"mtime_ns":1786963737772992800,"word_count":701},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\kv_event\\kv_event_publisher.cpp":{"size":12755,"mtime_ns":1786963737785618600,"word_count":875},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\local_hot_cache.cpp":{"size":19884,"mtime_ns":1786963737785618600,"word_count":1857},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master.cpp":{"size":88503,"mtime_ns":1788441619317981400,"word_count":5518},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master_admin_service.cpp":{"size":50115,"mtime_ns":1786963737785618600,"word_count":3061},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master_client.cpp":{"size":88040,"mtime_ns":1788421144249638700,"word_count":6012},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master_metric_manager.cpp":{"size":111439,"mtime_ns":1786963737785618600,"word_count":6128},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master_service.cpp":{"size":575007,"mtime_ns":1788441910616988000,"word_count":38297},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master_snapshot_manager.cpp":{"size":25216,"mtime_ns":1786963737788969700,"word_count":1810},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\master_snapshot_repository.cpp":{"size":13342,"mtime_ns":1786976645788931900,"word_count":883},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\memory_alloc.cpp":{"size":383,"mtime_ns":1781678985212026200,"word_count":31},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\metadata_store.cpp":{"size":1792,"mtime_ns":1786963737788969700,"word_count":108},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\mmap_arena.cpp":{"size":11119,"mtime_ns":1781678985212026200,"word_count":1152},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\offset_allocator.cpp":{"size":26141,"mtime_ns":1786963737788969700,"word_count":2404},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\partition\\partition_router.cpp":{"size":2990,"mtime_ns":1788421144252639700,"word_count":263},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\posix_file.cpp":{"size":4326,"mtime_ns":1786963737788969700,"word_count":398},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\real_client.cpp":{"size":321095,"mtime_ns":1788421144253639800,"word_count":24409},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\real_client_main.cpp":{"size":7930,"mtime_ns":1786976645789945900,"word_count":404},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\registered_pinned_memory.cpp":{"size":8695,"mtime_ns":1786976645791397500,"word_count":776},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\registered_pinned_memory.h":{"size":2274,"mtime_ns":1786963737788969700,"word_count":172},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\rpc_service.cpp":{"size":97438,"mtime_ns":1788421144646385700,"word_count":5584},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\segment.cpp":{"size":69573,"mtime_ns":1788008058575348000,"word_count":4734},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\serialize\\serializer.cpp":{"size":42113,"mtime_ns":1786976645792398900,"word_count":2631},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\shm_helper.cpp":{"size":5767,"mtime_ns":1786963737788969700,"word_count":511},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\spdk\\CMakeLists.txt":{"size":143,"mtime_ns":1781678985215033200,"word_count":6},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\spdk\\spdk_wrapper.cpp":{"size":16105,"mtime_ns":1781678985215033200,"word_count":1297},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\ssd_register_client.cpp":{"size":5600,"mtime_ns":1781678985215033200,"word_count":527},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\standby_state_machine.cpp":{"size":11520,"mtime_ns":1781678985215033200,"word_count":728},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\storage\\distributed\\distributed_storage_backend.cpp":{"size":12704,"mtime_ns":1786976645793399200,"word_count":990},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\storage\\distributed\\hf3fs_adapter.cpp":{"size":12277,"mtime_ns":1781678985216035500,"word_count":1161},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\storage_backend.cpp":{"size":257419,"mtime_ns":1788421144256353600,"word_count":20971},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\store_c.cpp":{"size":11938,"mtime_ns":1781678985217035500,"word_count":1147},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\store_c_shared.cpp":{"size":940,"mtime_ns":1786963737788969700,"word_count":140},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\task_manager.cpp":{"size":16374,"mtime_ns":1781678985217541100,"word_count":1137},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\tenant_quota.cpp":{"size":14767,"mtime_ns":1786963737788969700,"word_count":1115},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\tenant_quota_policy_store.cpp":{"size":15822,"mtime_ns":1786963737788969700,"word_count":1447},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\thread_pool.cpp":{"size":1143,"mtime_ns":1781678985217541100,"word_count":87},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\transfer_task.cpp":{"size":58102,"mtime_ns":1786976645795398900,"word_count":4785},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\types.cpp":{"size":6149,"mtime_ns":1788421144257354700,"word_count":300},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\uds_transport.cpp":{"size":10398,"mtime_ns":1786963737788969700,"word_count":1012},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\uring_file.cpp":{"size":30791,"mtime_ns":1786976645795398900,"word_count":2696},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\utils.cpp":{"size":28970,"mtime_ns":1786976645796544700,"word_count":2897},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\utils\\file_util.cpp":{"size":4547,"mtime_ns":1781678985218544000,"word_count":387},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\utils\\s3_helper.cpp":{"size":31051,"mtime_ns":1786963737788969700,"word_count":2370},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\utils\\type_util.cpp":{"size":1326,"mtime_ns":1786976645796544700,"word_count":159},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_allocation_strategy.cpp":{"size":5666,"mtime_ns":1788421144257354700,"word_count":416},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_client.cpp":{"size":12880,"mtime_ns":1788421144257354700,"word_count":885},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_config.cpp":{"size":866,"mtime_ns":1788421144257354700,"word_count":89},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_control_plane.cpp":{"size":4311,"mtime_ns":1788421144258354800,"word_count":274},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_master_manager.cpp":{"size":14332,"mtime_ns":1788421144258354800,"word_count":1095},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_metadata.cpp":{"size":6723,"mtime_ns":1788421144258354800,"word_count":491},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_metadata_store.cpp":{"size":7636,"mtime_ns":1788421144259354700,"word_count":586},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_metrics.cpp":{"size":2383,"mtime_ns":1788421144259354700,"word_count":175},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\src\\vchunk_transfer_engine.cpp":{"size":5609,"mtime_ns":1788421144259354700,"word_count":401},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\CMakeLists.txt":{"size":12942,"mtime_ns":1788421144307204800,"word_count":432},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\allocation_strategy_test.cpp":{"size":44013,"mtime_ns":1786963737788969700,"word_count":3531},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\batch_remove_test.cpp":{"size":13444,"mtime_ns":1781678985220549600,"word_count":1098},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\buffer_allocator_test.cpp":{"size":24161,"mtime_ns":1786976645797603000,"word_count":1794},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\client_buffer_test.cpp":{"size":15733,"mtime_ns":1786963737788969700,"word_count":1425},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\client_integration_test.cpp":{"size":82614,"mtime_ns":1788421144260421400,"word_count":6516},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\client_local_hot_cache_test.cpp":{"size":51492,"mtime_ns":1786963737802609400,"word_count":4775},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\client_metrics_test.cpp":{"size":21394,"mtime_ns":1786963737802609400,"word_count":1354},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\client_storage_backend_test.cpp":{"size":1748,"mtime_ns":1786963737802609400,"word_count":129},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\client_tcp_local_memcpy_test.cpp":{"size":18159,"mtime_ns":1781678985221553100,"word_count":1172},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\cxl_client_integration_test.cpp":{"size":19315,"mtime_ns":1781678985222557200,"word_count":1433},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\deadline_scheduler_test.cpp":{"size":7378,"mtime_ns":1786963737802609400,"word_count":501},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\distributed_object_store_provider.py":{"size":1083,"mtime_ns":1781678985222557200,"word_count":83},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\dummy_client_get_buffer_test.cpp":{"size":26965,"mtime_ns":1786963737802609400,"word_count":2357},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\CMakeLists.txt":{"size":2928,"mtime_ns":1788421144261423100,"word_count":148},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\chaos_rand_test.cpp":{"size":14954,"mtime_ns":1781678985223561200,"word_count":1385},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\chaos_test.cpp":{"size":13407,"mtime_ns":1781678985224564800,"word_count":1203},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\chaosctl.cpp":{"size":9795,"mtime_ns":1781678985224564800,"word_count":874},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\client_ctl_cases\\case1.txt":{"size":211,"mtime_ns":1781678985224564800,"word_count":43},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\client_ctl_cases\\case2.txt":{"size":617,"mtime_ns":1781678985224564800,"word_count":87},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\client_runner.cpp":{"size":7008,"mtime_ns":1781678985225567500,"word_count":702},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\client_wrapper.cpp":{"size":12085,"mtime_ns":1786963737804326200,"word_count":962},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\client_wrapper.h":{"size":4287,"mtime_ns":1786963737804326200,"word_count":449},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\clientctl.cpp":{"size":9623,"mtime_ns":1786963737804326200,"word_count":913},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\e2e_rand_test.cpp":{"size":6986,"mtime_ns":1781678985226570000,"word_count":648},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\e2e_utils.h":{"size":2801,"mtime_ns":1781678985226570000,"word_count":225},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\gc_e2e_test.cpp":{"size":22595,"mtime_ns":1788421144261423100,"word_count":2181},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\oplog_batch_e2e_test.cpp":{"size":3002,"mtime_ns":1786963737804326200,"word_count":201},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\oplog_fault_ctl.sh":{"size":3224,"mtime_ns":1786963737804326200,"word_count":394},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\oplog_fault_ctl_test.sh":{"size":1949,"mtime_ns":1786963737804326200,"word_count":209},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\oplog_ha_client.cpp":{"size":10776,"mtime_ns":1786963737804326200,"word_count":1031},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\process_handler.cpp":{"size":18424,"mtime_ns":1786963737804326200,"word_count":1758},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\process_handler.h":{"size":6583,"mtime_ns":1786963737804326200,"word_count":714},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\readme.md":{"size":7255,"mtime_ns":1781678985227573000,"word_count":927},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\run_nof_heartbeat_tcp_e2e.sh":{"size":8942,"mtime_ns":1781678985227573000,"word_count":812},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\run_oplog_batch_cluster.sh":{"size":71362,"mtime_ns":1786963737804326200,"word_count":6265},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\run_oplog_batch_cluster_test.sh":{"size":5805,"mtime_ns":1786963737804326200,"word_count":553},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\storage_backend_e2e_test.cpp":{"size":14871,"mtime_ns":1781678985227573000,"word_count":1297},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\e2e\\store_client_e2e.py":{"size":3713,"mtime_ns":1781678985227573000,"word_count":291},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\eviction_strategy_test.cpp":{"size":4268,"mtime_ns":1781678985228575500,"word_count":340},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\file_storage_promotion_test.cpp":{"size":17038,"mtime_ns":1781678985228575500,"word_count":1419},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\file_storage_test.cpp":{"size":32959,"mtime_ns":1786976645798653700,"word_count":1951},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\file_util_test.cpp":{"size":4144,"mtime_ns":1781678985228575500,"word_count":330},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\common\\redis\\redis_test_utils.h":{"size":1443,"mtime_ns":1781678985229577800,"word_count":105},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\backends\\k8s\\high_availability_k8s_test.cpp":{"size":12997,"mtime_ns":1781678985229577800,"word_count":854},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\backends\\redis\\high_availability_redis_test.cpp":{"size":8369,"mtime_ns":1781678985230577500,"word_count":428},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\etcd_leader_hang_e2e.sh":{"size":5061,"mtime_ns":1786963737804326200,"word_count":432},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\ha_backend_availability_test.cpp":{"size":1974,"mtime_ns":1786963737804326200,"word_count":94},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\high_availability_test.cpp":{"size":32969,"mtime_ns":1786963737804326200,"word_count":2154},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\high_availability_test_fixture.h":{"size":289,"mtime_ns":1781678985231083700,"word_count":32},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\leadership\\leader_label_reconciler_test.cpp":{"size":5653,"mtime_ns":1786963737804326200,"word_count":373},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\master_service_ha_test.cpp":{"size":169122,"mtime_ns":1788421144262422900,"word_count":8402},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\mock_metadata_store.h":{"size":3762,"mtime_ns":1786976645799655300,"word_count":328},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\mock_snapshot_provider.h":{"size":1552,"mtime_ns":1786963737804326200,"word_count":122},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_applier_test.cpp":{"size":21124,"mtime_ns":1788008058577713700,"word_count":1393},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_batch_auditor_test.cpp":{"size":6698,"mtime_ns":1786963737804326200,"word_count":474},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_batch_codec_test.cpp":{"size":11961,"mtime_ns":1786963737804326200,"word_count":697},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_batch_standby_reader_test.cpp":{"size":19562,"mtime_ns":1786963737804326200,"word_count":1081},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_batch_storage_test.cpp":{"size":19583,"mtime_ns":1786976645800729500,"word_count":1099},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_test_failpoint_test.cpp":{"size":2343,"mtime_ns":1786963737804326200,"word_count":160},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\oplog_types_test.cpp":{"size":731,"mtime_ns":1786963737804326200,"word_count":46},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\oplog\\ordered_oplog_writer_test.cpp":{"size":34474,"mtime_ns":1786976645800729500,"word_count":1962},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\catalog\\backends\\embedded\\embedded_snapshot_catalog_store_test.cpp":{"size":10555,"mtime_ns":1781678985235623400,"word_count":535},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\catalog\\backends\\redis\\redis_snapshot_catalog_store_test.cpp":{"size":8911,"mtime_ns":1781678985235623400,"word_count":494},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\catalog_backed_snapshot_provider_test.cpp":{"size":10540,"mtime_ns":1786963737804326200,"word_count":669},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\master_service_promotion_test_for_snapshot.cpp":{"size":8793,"mtime_ns":1786963737804326200,"word_count":752},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\master_service_ssd_test_for_snapshot.cpp":{"size":20221,"mtime_ns":1786963737804326200,"word_count":1420},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\master_service_test_for_snapshot.cpp":{"size":180575,"mtime_ns":1786976645801828600,"word_count":13831},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\master_service_test_for_snapshot_base.h":{"size":35000,"mtime_ns":1786976645802828400,"word_count":2922},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\master_snapshot_codec_test.cpp":{"size":8747,"mtime_ns":1786976645802828400,"word_count":707},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\object\\backends\\local\\local_file_snapshot_object_store_test.cpp":{"size":5200,"mtime_ns":1781678985238631600,"word_count":338},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\snapshot_child_process_test.cpp":{"size":57516,"mtime_ns":1786976645802828400,"word_count":3627},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\snapshot\\snapshot_test_utils.h":{"size":14692,"mtime_ns":1786963737804326200,"word_count":1007},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\standby\\ha_metric_manager_test.cpp":{"size":8322,"mtime_ns":1786963737818801700,"word_count":462},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\standby\\hot_standby_service_test.cpp":{"size":35627,"mtime_ns":1788421144263423100,"word_count":2327},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\standby\\hot_standby_snapshot_bootstrap_test.cpp":{"size":9842,"mtime_ns":1788421144263423100,"word_count":520},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ha\\standby\\standby_state_machine_test.cpp":{"size":27128,"mtime_ns":1781678985240635800,"word_count":1396},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\health_check_test.cpp":{"size":8991,"mtime_ns":1781678985240635800,"word_count":668},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\host_port_fix_test.cpp":{"size":1619,"mtime_ns":1786963737819303600,"word_count":148},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\http_metadata_server_test.cpp":{"size":3730,"mtime_ns":1786963737820332700,"word_count":270},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\io_pattern_framework_test.cpp":{"size":71756,"mtime_ns":1788426265819429000,"word_count":4116},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ipv6_client_test.cpp":{"size":16051,"mtime_ns":1781678985240635800,"word_count":1489},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\kv_event_publisher_test.cpp":{"size":7605,"mtime_ns":1786963737820332700,"word_count":546},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\master_admin_server_test.cpp":{"size":48242,"mtime_ns":1786963737820332700,"word_count":2794},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\master_metrics_test.cpp":{"size":46882,"mtime_ns":1786963737820332700,"word_count":2756},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\master_service_config_test.cpp":{"size":2716,"mtime_ns":1788421144647877600,"word_count":116},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\master_service_ssd_test.cpp":{"size":39181,"mtime_ns":1786963737820332700,"word_count":2962},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\master_service_tenant_quota_test.cpp":{"size":36942,"mtime_ns":1788008058578788900,"word_count":1985},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\master_service_test.cpp":{"size":349866,"mtime_ns":1788421144264275800,"word_count":24221},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\mmap_arena_fallback_test.cpp":{"size":7309,"mtime_ns":1786963737820332700,"word_count":624},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\mmap_arena_test.cpp":{"size":25793,"mtime_ns":1786963737820332700,"word_count":2726},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\mutex_test.cpp":{"size":7305,"mtime_ns":1781678985243641600,"word_count":595},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\nof_heartbeat_test.cpp":{"size":10163,"mtime_ns":1781678985243641600,"word_count":522},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\non_ha_reconnect_test.cpp":{"size":6743,"mtime_ns":1781678985243641600,"word_count":464},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\object_data_type_test.cpp":{"size":5636,"mtime_ns":1786963737820332700,"word_count":395},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\offload_on_evict_test.cpp":{"size":17359,"mtime_ns":1786963737820332700,"word_count":1501},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\offset_allocator_test.cpp":{"size":66004,"mtime_ns":1786963737820332700,"word_count":5833},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\partition_router_test.cpp":{"size":3066,"mtime_ns":1788421144264275800,"word_count":170},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\posix_file_test.cpp":{"size":5847,"mtime_ns":1781678985244643700,"word_count":529},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\promotion_on_hit_test.cpp":{"size":119415,"mtime_ns":1788421144264275800,"word_count":10724},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\pybind_client_test.cpp":{"size":76773,"mtime_ns":1786963737820332700,"word_count":5711},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\registered_pinned_memory_test.cpp":{"size":3692,"mtime_ns":1786963737820332700,"word_count":299},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\replica_selection_test.cpp":{"size":13142,"mtime_ns":1786963737820332700,"word_count":1029},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\rpc_timeout_test.cpp":{"size":4399,"mtime_ns":1786963737820332700,"word_count":471},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\runtime_accelerator_test.cpp":{"size":3894,"mtime_ns":1786963737820332700,"word_count":294},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\scripts\\setup_dev.sh":{"size":569,"mtime_ns":1781678985245645900,"word_count":77},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\segment_test.cpp":{"size":30651,"mtime_ns":1788008058579789400,"word_count":2197},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\serializer_test.cpp":{"size":7596,"mtime_ns":1786963737820332700,"word_count":549},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\slot_hash_test.cpp":{"size":5527,"mtime_ns":1788421144264275800,"word_count":474},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\ssd_metrics_test.cpp":{"size":16705,"mtime_ns":1781678985246648300,"word_count":1247},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\storage_backend_test.cpp":{"size":206575,"mtime_ns":1788421144264275800,"word_count":14285},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\stress_cluster_benchmark.py":{"size":27143,"mtime_ns":1781678985247650000,"word_count":2199},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\stress_workload_test.cpp":{"size":15129,"mtime_ns":1781678985247650000,"word_count":1413},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\task_executor_test.cpp":{"size":17776,"mtime_ns":1781678985247650000,"word_count":1297},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\task_integration_test.cpp":{"size":57381,"mtime_ns":1781678985248651700,"word_count":4871},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\task_manager_test.cpp":{"size":17356,"mtime_ns":1781678985248651700,"word_count":1060},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\tenant_id_test.cpp":{"size":2200,"mtime_ns":1786963737820332700,"word_count":129},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\tenant_quota_test.cpp":{"size":24636,"mtime_ns":1786963737820332700,"word_count":1464},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\test_server_helpers.h":{"size":10528,"mtime_ns":1786963737820332700,"word_count":667},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\thread_pool_test.cpp":{"size":3903,"mtime_ns":1781678985248651700,"word_count":324},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\transfer_task_test.cpp":{"size":7130,"mtime_ns":1786976645807825700,"word_count":589},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\uds_transport_test.cpp":{"size":8556,"mtime_ns":1786963737820332700,"word_count":591},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\utils\\common.h":{"size":2599,"mtime_ns":1781678985249670600,"word_count":185},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\utils_test.cpp":{"size":9467,"mtime_ns":1786976645807825700,"word_count":838},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_allocation_strategy_test.cpp":{"size":4432,"mtime_ns":1788421144264275800,"word_count":295},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_client_test.cpp":{"size":16169,"mtime_ns":1788421144264275800,"word_count":1046},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_config_test.cpp":{"size":2160,"mtime_ns":1788421144264275800,"word_count":134},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_master_manager_test.cpp":{"size":8285,"mtime_ns":1788421144264275800,"word_count":488},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_master_service_test.cpp":{"size":6013,"mtime_ns":1788421144264275800,"word_count":314},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_metadata_store_test.cpp":{"size":10982,"mtime_ns":1788421144264275800,"word_count":652},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_metadata_test.cpp":{"size":5231,"mtime_ns":1788421144264275800,"word_count":290},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_test_allocator.h":{"size":1843,"mtime_ns":1788421144264275800,"word_count":156},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\vchunk_transfer_engine_test.cpp":{"size":3477,"mtime_ns":1788421144264275800,"word_count":264},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tests\\zstd_util_test.cpp":{"size":3045,"mtime_ns":1781678985249670600,"word_count":238},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tools\\CMakeLists.txt":{"size":384,"mtime_ns":1786963737820332700,"word_count":10},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tools\\oplog_batch_auditor.cpp":{"size":9284,"mtime_ns":1786963737820332700,"word_count":765},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tools\\oplog_batch_auditor.h":{"size":1369,"mtime_ns":1786963737820332700,"word_count":85},"C:\\workspace\\code\\opensource\\Mooncake\\mooncake-store\\tools\\oplog_batch_inspector.cpp":{"size":8158,"mtime_ns":1786963737820332700,"word_count":731}}
\ No newline at end of file
diff --git a/mooncake-store/include/admission_ops.h b/mooncake-store/include/admission_ops.h
new file mode 100644
index 0000000000..9ef1ade825
--- /dev/null
+++ b/mooncake-store/include/admission_ops.h
@@ -0,0 +1,3 @@
+#pragma once
+
+#include "io_pattern/ops.h"
diff --git a/mooncake-store/include/cfm_client.h b/mooncake-store/include/cfm_client.h
new file mode 100644
index 0000000000..c94a0b0929
--- /dev/null
+++ b/mooncake-store/include/cfm_client.h
@@ -0,0 +1,3 @@
+#pragma once
+
+#include "io_pattern/client.h"
diff --git a/mooncake-store/include/cfm_client_impl.h b/mooncake-store/include/cfm_client_impl.h
new file mode 100644
index 0000000000..da6bbcfe9c
--- /dev/null
+++ b/mooncake-store/include/cfm_client_impl.h
@@ -0,0 +1,2 @@
+#pragma once
+#include "io_pattern/cfm_client_impl.h"
diff --git a/mooncake-store/include/client_service.h b/mooncake-store/include/client_service.h
index 631e4328bd..108996792a 100644
--- a/mooncake-store/include/client_service.h
+++ b/mooncake-store/include/client_service.h
@@ -908,8 +908,30 @@ class Client {
std::mutex leader_switch_mutex_;
std::optional current_master_view_;
std::string direct_master_address_;
+ // Etcd endpoints parsed from the HA backend entry, used to load the KV
+ // partition routing snapshot. Empty in non-HA mode.
+ std::string etcd_connstring_;
+ // Resolved cluster namespace (cluster_id) cached after the first routing
+ // load; reused by the periodic routing refresh loop.
+ std::string cluster_id_;
+ // CVM multi-submaster mode: set when the client bootstraps from the
+ // /cvm//masters/ registry (no single-leader master_view in etcd).
+ // Empty in single-leader HA mode and direct mode. Also drives the
+ // ping-failure re-discovery path.
+ std::string cvm_cluster_namespace_;
+ // CVM 多 submaster:已发现的全部 primary submaster 地址(含当前连接的
+ // 那个)。用于 client 侧全量 mount(同一 segment 挂到所有 submaster)与
+ // 多 submaster 心跳(防止各 submaster 因收不到 ping 而误卸载 segment)。
+ // 由 ConnectToCvmSubmasters 填充。
+ std::mutex cvm_submaster_addresses_mutex_;
+ std::vector cvm_submaster_addresses_;
+ // 串行化 TryLoadRoutingOnce(及其内部对 cluster_id_ 的读写),
+ // 用于 routing-refresh 线程与 SLOT_NOT_OWNED 触发的按需刷新之间。
+ std::mutex routing_load_mutex_;
std::thread leader_monitor_thread_;
std::atomic leader_monitor_running_{false};
+ std::thread routing_refresh_thread_;
+ std::atomic routing_refresh_running_{false};
std::thread storage_heartbeat_thread_;
std::atomic storage_heartbeat_running_{false};
std::thread task_poll_thread_;
@@ -918,8 +940,25 @@ class Client {
std::atomic segment_desc_publish_pending_{false};
std::atomic rpc_meta_publish_pending_{false};
ErrorCode SwitchLeader(const ha::MasterView& target_view);
+ // CVM multi-submaster bootstrap/failover: connect the etcd store client,
+ // discover the cluster namespace from /cvm//masters/, and connect to
+ // the first live primary submaster (first-registered first, matching the
+ // CvmController ranking). On success sets cvm_cluster_namespace_.
+ ErrorCode ConnectToCvmSubmasters(const std::string& etcd_endpoints);
+ void LoadPartitionRouting();
+ void TryLoadRoutingOnce();
+ void RoutingRefreshThreadMain();
void LeaderMonitorThreadMain();
void StorageHeartbeatThreadMain();
+ // CVM 多 submaster:向所有 primary submaster 定向 Ping(不切换当前连接),
+ // 各 submaster 收到 ping 后会保活该 client;返回 NEED_REMOUNT 的 submaster
+ // 会被重新 mount(重新注册 segment)。单 submaster 失败不影响其他。
+ void HeartbeatAllSubmasters();
+ // CVM 多 submaster:重新扫描 /cvm//masters/ 注册表,更新已发现的
+ // primary 地址列表;对新增的 submaster 全量 mount 所有本地 segment,使
+ // 新加入的 submaster 也具备给本 client 分配副本的能力。由路由刷新线程
+ // 周期调用。
+ void RefreshSubmasterAddresses();
void TaskPollThreadMain();
void EnsureStorageControlPlaneStarted();
void PollAndDispatchTasks();
diff --git a/mooncake-store/include/collector_impl.h b/mooncake-store/include/collector_impl.h
new file mode 100644
index 0000000000..f85dc274e6
--- /dev/null
+++ b/mooncake-store/include/collector_impl.h
@@ -0,0 +1,2 @@
+#pragma once
+#include "io_pattern/collector_impl.h"
diff --git a/mooncake-store/include/cvm/cvm_controller.h b/mooncake-store/include/cvm/cvm_controller.h
new file mode 100644
index 0000000000..b581cc0980
--- /dev/null
+++ b/mooncake-store/include/cvm/cvm_controller.h
@@ -0,0 +1,163 @@
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "cvm/cvm_types.h"
+#include "mutex.h"
+#include "types.h"
+
+namespace mooncake {
+namespace cvm {
+
+class CvmServiceDelegate;
+class CvmHttpServer;
+
+// In-process control plane for the CVM (Cache View Master).
+//
+// Responsibilities (P1 skeleton):
+// - Register this master under an etcd lease (liveness + role).
+// - Load and continuously watch the KV view (slot -> primary master).
+// - Expose local read access to the cached view (OwnsSlot / GetKvView).
+//
+// The full slot-migration / failover / segment-view state machines are added
+// in later phases.
+class CvmController {
+ public:
+ struct Config {
+ std::string cluster_namespace;
+ std::string master_id;
+ std::string address; // RPC endpoint of this master.
+ MasterRole role = MasterRole::kPrimary;
+ int64_t registration_lease_ttl_sec = 3;
+
+ // CvmHttpServer(外部 HTTP 接口)配置;http_port == 0 表示不启动。
+ std::string http_host = "0.0.0.0";
+ uint16_t http_port = 0;
+
+ // 集群中允许同时 serving 的 submaster 上限(名额协调,先到先得)。
+ // 排名前 submaster_count 个 master 为 kPrimary,其余降级为 kStandby。
+ uint32_t submaster_count = 1;
+
+ // 视图快照生成(SyncOnce)的调度周期。
+ std::chrono::milliseconds sync_interval{5000};
+ };
+
+ explicit CvmController(Config config);
+ ~CvmController();
+
+ CvmController(const CvmController&) = delete;
+ CvmController& operator=(const CvmController&) = delete;
+
+ void SetDelegate(CvmServiceDelegate* delegate);
+
+ // 当前角色(随名额协调动态变化)。
+ MasterRole GetCurrentRole() const { return current_role_.load(); }
+
+ // 排名第一(先到先得)的 primary 的 RPC 地址,作为 standby 的单源回放
+ // 目标。当成员列表为空或本机即为排名第一的 primary 时返回空字符串。
+ std::string GetPrimaryAddress();
+
+ // 排名前 submaster_count 的 primary(submaster)成员(含 master_id 与
+ // address),作为 standby 的多源回放目标。排除本机;成员列表为空或本机
+ // 覆盖全部 primary 名额时返回空列表。
+ std::vector GetPrimaryPeers();
+
+ // standby 动态绑定(2c):按「本 standby 负责的 slot 区间」过滤出拥有这些
+ // slot 的 primary,作为回放源(而非回放全部 primary)。本机为 primary 或
+ // 无法确定负责区间时返回空列表。
+ std::vector GetBindingSources();
+
+ ErrorCode Start();
+ void Stop();
+
+ // 调度 EtcdViewStore 聚合原始记录 -> 生成视图快照 -> 回写 etcd。
+ ErrorCode SyncOnce();
+
+ // 把最新视图快照路径推送给 CvmHttpServer。
+ void PushViewPaths();
+
+ // etcd lease id backing this master's registration. Callers may reuse it
+ // for their own records (slot/segment ownership) so they share the same
+ // lifecycle and are auto-removed on master death. 0 until Start() succeeds.
+ EtcdLeaseId GetLeaseId() const { return lease_id_; }
+
+ // Whether this master currently owns `slot` as primary.
+ bool OwnsSlot(uint16_t slot) const;
+
+ // Snapshot of the cached KV view and its version.
+ std::vector GetKvView() const;
+ ViewVersionId GetKvViewVersion() const;
+
+ private:
+ struct WatchState {
+ std::mutex mutex;
+ std::condition_variable cv;
+ bool dirty = false;
+ bool broken = false;
+ };
+
+ Config config_;
+ CvmServiceDelegate* delegate_ = nullptr;
+
+ mutable SharedMutex view_mutex_;
+ std::unordered_map kv_view_;
+ ViewVersionId kv_view_version_{0};
+
+ EtcdLeaseId lease_id_{0};
+ std::atomic running_{false};
+ std::atomic watch_armed_{false};
+ std::atomic masters_watch_armed_{false};
+
+ // 当前角色(随名额协调动态变化);初始为启动配置的 role。
+ std::atomic current_role_{MasterRole::kPrimary};
+
+ std::unique_ptr watch_state_;
+ std::unique_ptr masters_watch_state_;
+ std::thread watch_thread_;
+ std::thread masters_watch_thread_;
+ std::thread keepalive_thread_;
+ std::thread sync_thread_;
+ std::thread membership_thread_;
+
+ std::mutex sync_mutex_;
+ std::condition_variable sync_cv_;
+
+ std::mutex membership_mutex_;
+ std::condition_variable membership_cv_;
+
+ // 视图类型 -> etcd 快照路径("kv_view"/"segment_view")。
+ std::unordered_map view_paths_;
+ std::unique_ptr http_server_;
+
+ ErrorCode RefreshKvView();
+ void CancelWatchAndWait();
+ void WatchLoop();
+ void MastersWatchLoop();
+ void KeepaliveLoop();
+ void SyncLoop();
+ void MembershipLoop();
+ // 重算并回写角色(先到先得排名);membership 轮询与 masters watch 回调
+ // 共用,用 CAS 去重,保证并发下仅一次迁移与通知。
+ void ReconcileRole();
+ MasterRole ComputeDesiredRole();
+ // 加载所有存活 master 并按「先到先得」排序(registered_at_ms 升序,
+ // tie-break master_id 字典序)。失败返回 false。
+ bool LoadRankedMembers(std::vector& out);
+
+ static void WatchCallback(void* ctx, const char* key, size_t key_size,
+ const char* value, size_t value_size,
+ int event_type, int64_t mod_revision);
+};
+
+} // namespace cvm
+} // namespace mooncake
diff --git a/mooncake-store/include/cvm/cvm_http_server.h b/mooncake-store/include/cvm/cvm_http_server.h
new file mode 100644
index 0000000000..13b22511ca
--- /dev/null
+++ b/mooncake-store/include/cvm/cvm_http_server.h
@@ -0,0 +1,61 @@
+#pragma once
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+#include "types.h"
+
+namespace mooncake {
+namespace cvm {
+
+// CVM 三模块之一:外部 HTTP 接口(cvmhttpserver)。
+//
+// 提供只读 HTTP API,按 etcd 中的视图快照路径读取快照并返回 JSON,供网页 /
+// 外部客户端展示。视图快照路径由 CvmController 定期推送;未推送时使用基于
+// cluster_namespace 构造的默认快照路径(见 cvm_keys.h)。
+class CvmHttpServer {
+ public:
+ struct Config {
+ std::string host = "0.0.0.0";
+ uint16_t port = 0;
+ std::string cluster_namespace;
+ };
+
+ explicit CvmHttpServer(Config config);
+ ~CvmHttpServer();
+
+ CvmHttpServer(const CvmHttpServer&) = delete;
+ CvmHttpServer& operator=(const CvmHttpServer&) = delete;
+
+ ErrorCode Start();
+ void Stop();
+
+ // 供 CvmController 推送最新视图快照路径(etcd key)。
+ void SetKvViewSnapshotKey(const std::string& key);
+ void SetSegmentViewSnapshotKey(const std::string& key);
+
+ // 按当前路径读取快照并返回 JSON 字符串;路径为空或读取失败时返回空串。
+ std::string GetKvViewJson() const;
+ std::string GetSegmentViewJson() const;
+
+ private:
+ void InitRoutes();
+ std::string ReadSnapshot(const std::string& key) const;
+
+ Config config_;
+ std::unique_ptr server_;
+
+ mutable std::mutex path_mutex_;
+ std::string kv_view_snapshot_key_;
+ std::string segment_view_snapshot_key_;
+
+ std::atomic running_{false};
+};
+
+} // namespace cvm
+} // namespace mooncake
diff --git a/mooncake-store/include/cvm/cvm_keys.h b/mooncake-store/include/cvm/cvm_keys.h
new file mode 100644
index 0000000000..0e1163ecc9
--- /dev/null
+++ b/mooncake-store/include/cvm/cvm_keys.h
@@ -0,0 +1,100 @@
+#pragma once
+
+#include
+#include
+#include
+#include
+
+namespace mooncake {
+namespace cvm {
+
+// Root prefix for all CVM keys in etcd.
+inline constexpr std::string_view kCvmRootPrefix = "/cvm/";
+
+// End key for an etcd range scan over [prefix, PrefixEnd(prefix)).
+inline std::string PrefixEnd(std::string prefix) {
+ for (int i = static_cast(prefix.size()) - 1; i >= 0; --i) {
+ unsigned char c = static_cast(prefix[i]);
+ if (c < 0xFF) {
+ prefix[i] = static_cast(c + 1);
+ prefix.resize(i + 1);
+ return prefix;
+ }
+ }
+ return std::string(1, '\0');
+}
+
+// "/cvm//"
+inline std::string CvmNamespaceRoot(const std::string& cluster_namespace) {
+ return std::string(kCvmRootPrefix) + cluster_namespace + "/";
+}
+
+// "/cvm//kv_view/"
+inline std::string KvViewPrefix(const std::string& cluster_namespace) {
+ return CvmNamespaceRoot(cluster_namespace) + "kv_view/";
+}
+
+// "/cvm//kv_view/slot/"
+inline std::string SlotOwnerKey(const std::string& cluster_namespace,
+ uint16_t slot) {
+ char buf[16];
+ std::snprintf(buf, sizeof(buf), "%05u", static_cast(slot));
+ return KvViewPrefix(cluster_namespace) + "slot/" + buf;
+}
+
+// "/cvm//slot_meta/"
+inline std::string SlotMetadataExportPrefix(
+ const std::string& cluster_namespace) {
+ return CvmNamespaceRoot(cluster_namespace) + "slot_meta/";
+}
+
+// "/cvm//slot_meta/"
+// Binary (struct_pack) value holding the object metadata exported by the
+// previous live primary owner during a slot handoff.
+inline std::string SlotMetadataExportKey(const std::string& cluster_namespace,
+ uint16_t slot) {
+ char buf[16];
+ std::snprintf(buf, sizeof(buf), "%05u", static_cast(slot));
+ return SlotMetadataExportPrefix(cluster_namespace) + buf;
+}
+
+// "/cvm//segment_view/"
+inline std::string SegmentViewPrefix(const std::string& cluster_namespace) {
+ return CvmNamespaceRoot(cluster_namespace) + "segment_view/";
+}
+
+// "/cvm//segment_view/"
+inline std::string SegmentOwnerKey(const std::string& cluster_namespace,
+ const std::string& segment_id) {
+ return SegmentViewPrefix(cluster_namespace) + segment_id;
+}
+
+// "/cvm//masters/"
+inline std::string MasterRegistrationPrefix(
+ const std::string& cluster_namespace) {
+ return CvmNamespaceRoot(cluster_namespace) + "masters/";
+}
+
+// "/cvm//masters/"
+inline std::string MasterRegistrationKey(const std::string& cluster_namespace,
+ const std::string& master_id) {
+ return MasterRegistrationPrefix(cluster_namespace) + master_id;
+}
+
+// "/cvm//snapshot/"
+inline std::string SnapshotPrefix(const std::string& cluster_namespace) {
+ return CvmNamespaceRoot(cluster_namespace) + "snapshot/";
+}
+
+// "/cvm//snapshot/kv_view"
+inline std::string KvViewSnapshotKey(const std::string& cluster_namespace) {
+ return SnapshotPrefix(cluster_namespace) + "kv_view";
+}
+
+// "/cvm//snapshot/segment_view"
+inline std::string SegmentViewSnapshotKey(const std::string& cluster_namespace) {
+ return SnapshotPrefix(cluster_namespace) + "segment_view";
+}
+
+} // namespace cvm
+} // namespace mooncake
diff --git a/mooncake-store/include/cvm/cvm_service_delegate.h b/mooncake-store/include/cvm/cvm_service_delegate.h
new file mode 100644
index 0000000000..81c4c65208
--- /dev/null
+++ b/mooncake-store/include/cvm/cvm_service_delegate.h
@@ -0,0 +1,36 @@
+#pragma once
+
+#include
+
+#include "cvm/cvm_types.h"
+
+namespace mooncake {
+namespace cvm {
+
+// Callback interface implemented by MasterService. CvmController calls into
+// this delegate to notify the embedding master of slot ownership changes,
+// without taking a dependency on the concrete MasterService type.
+class CvmServiceDelegate {
+ public:
+ virtual ~CvmServiceDelegate() = default;
+
+ // Called when this master becomes the stable primary owner of `slot`.
+ virtual void OnSlotAcquired(uint16_t slot) = 0;
+
+ // Called when this master releases ownership of `slot`.
+ virtual void OnSlotReleased(uint16_t slot) = 0;
+
+ // Called when the CVM membership coordinator decides this master's role
+ // should change (e.g. demoted to standby because the submaster quota is
+ // full, or promoted back to primary). MasterService reacts by switching
+ // its serving/standby state machine accordingly.
+ virtual void OnRoleChanged(MasterRole new_role) = 0;
+
+ // Called when the cached slot->primary view changes (slot ownership
+ // rebalanced or a primary's lease expired). A standby uses this to re-bind
+ // its replay sources even when its own role stays kStandby.
+ virtual void OnKvViewChanged() {}
+};
+
+} // namespace cvm
+} // namespace mooncake
diff --git a/mooncake-store/include/cvm/cvm_types.h b/mooncake-store/include/cvm/cvm_types.h
new file mode 100644
index 0000000000..ea5d71b084
--- /dev/null
+++ b/mooncake-store/include/cvm/cvm_types.h
@@ -0,0 +1,108 @@
+#pragma once
+
+#include
+#include
+#include
+
+#include "types.h"
+
+namespace mooncake {
+namespace cvm {
+
+// ---------------------------------------------------------------------------
+// Enumerations
+// ---------------------------------------------------------------------------
+
+// Ownership state of a logical KV slot.
+enum class SlotState : int32_t {
+ kStable = 0, // Served by primary_master_id.
+ kMigrating = 1, // Handing off to migrating_to_master_id.
+};
+
+// Role of a master within the CVM topology.
+enum class MasterRole : int32_t {
+ kPrimary = 0,
+ kStandby = 1,
+};
+
+// Ownership state of a segment (segment view is reserved).
+enum class SegmentOwnerState : int32_t {
+ kStable = 0,
+ kTransitioning = 1,
+};
+
+// ---------------------------------------------------------------------------
+// KV view: slot -> primary master ownership
+// ---------------------------------------------------------------------------
+
+// NOTE: enum fields are stored as int32_t so the records serialize/deserialize
+// identically across languages and compiler settings.
+struct SlotOwner {
+ uint16_t slot{0};
+ std::string primary_master_id;
+ int32_t state{0}; // SlotState
+ std::string migrating_to_master_id; // empty when stable
+};
+YLT_REFL(SlotOwner, slot, primary_master_id, state, migrating_to_master_id);
+
+// ---------------------------------------------------------------------------
+// Segment view: segment -> owner master ownership (reserved)
+// ---------------------------------------------------------------------------
+
+struct SegmentOwner {
+ std::string segment_id;
+ std::string owner_master_id;
+ int32_t state{0}; // SegmentOwnerState
+};
+YLT_REFL(SegmentOwner, segment_id, owner_master_id, state);
+
+// ---------------------------------------------------------------------------
+// Master registration: liveness + role, persisted under an etcd lease
+// ---------------------------------------------------------------------------
+
+struct MasterRegistration {
+ std::string master_id;
+ std::string address; // RPC endpoint, e.g. "host:port"
+ int32_t role{0}; // MasterRole
+ int64_t registered_at_ms{0};
+};
+YLT_REFL(MasterRegistration, master_id, address, role, registered_at_ms);
+
+// ---------------------------------------------------------------------------
+// Aggregated views (in-memory snapshots)
+// ---------------------------------------------------------------------------
+
+struct KvView {
+ std::vector