Skip to content

fix(pd): handle deep prompt-cache keys iteratively - #1580

Closed
WANDY666 wants to merge 1 commit into
mainfrom
fix/pd-prompt-cache-recursion
Closed

WANDY666 wants to merge 1 commit into
mainfrom
fix/pd-prompt-cache-recursion

Conversation

@WANDY666

Copy link
Copy Markdown
Contributor

Summary

  • replace recursive prompt-cache trie insertion and matching with iterative traversal
  • remove the process-wide sys.setrecursionlimit configuration
  • backport the existing support_ds4 fix from commit 7bc0955b20

Deep sampled prompt-cache keys can still exceed any fixed Python recursion limit. Iterative traversal removes that limit without changing the trie/LRU behavior.

Validation

  • python -m pytest -q -p no:cacheprovider unit_tests/server/test_pd_cache_aware.py — 12 passed
  • inserted and matched a 10,000-character key with Python's recursion limit left at 1,000
  • black --check --line-length=120 on both changed files
  • verified the backport patch-id matches the original commit

@hiworldwzj hiworldwzj closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants