fix: escape glob metacharacters in SCAN match patterns - #702
Open
vishal-bala wants to merge 2 commits into
Open
Conversation
SCAN/KEYS MATCH patterns are glob-style, but several call sites interpolated a caller-supplied literal straight into one. A cache name, index prefix or route name containing *, ?, [ or \ therefore produced a pattern matching keys the caller did not own -- and on the destructive paths those keys were then deleted or rewritten. Verified on Redis 8.4.5: SemanticCache(name="cache[ab]").clear() deletes every entry under cachea: and cacheb: while leaving its own keys intact, silently and with no error. The same shape reaches delete_route_references and the migration executor's RENAME/DUMP-RESTORE-DEL path. Patterns are now built with match_pattern(), which escapes each literal segment and appends the trailing glob. Exposing a builder rather than a bare escaper is deliberate: escaping stops being a step a future call site can forget, which matters because no lint rule can express the invariant (ruff and flake8 are unconfigured, mypy is not strict, and patterns are consumed several call sites away from where they are built). Covered sites: BaseCache.clear/aclear, SemanticRouter._route_pattern, build_scan_match_patterns (and through it executor, async_executor, validation, async_validation), and both planners' key sampling. Escaping rather than rejecting such names, because rejecting is a breaking change that orphans keys: a deployment whose cache is already named "cache[ab]" could no longer construct the object to clean up after itself. Tests split by what can actually answer each question. That RedisVL emits the right pattern is an exact-string assertion, needing no matcher. That Redis then interprets it as intended is asked of a real server: the existing cache regression in test_embedcache.py, and a new migration and router prefix case in test_migration_v1.py. No glob matcher is reimplemented anywhere, so nothing can drift from Redis. Two pre-existing issues in the same neighbourhood are deliberately left alone: BaseCache.clear/aclear never advance a Mapping cursor on Redis Cluster (fixed separately by replacing the hand-rolled loop with scan_iter), and async_planner appends the key separator where the other three sites do not, so it samples a narrower key set than the index covers.
Two things that surprise anyone assembling a least-privilege credential, both measured on Redis 8.4.5 rather than inferred. A cache read is not read-only: with a TTL configured, every hit refreshes the matched entries' TTL, so the read path issues EXPIRE, which is in @Write and not @READ. A lookup-only credential therefore fails on a cache hit rather than on the write that populated the entry. Granting the command is only half of it -- ACL DRYRUN shows +expire under a read-only key pattern such as %R~llmcache:* is denied on the key, and that is the shape the Key permissions table presents as sufficient for querying. Placed in "Roles built from @READ and @Write", where a reader assembling exactly that role will be. ACL key patterns are also glob-style, matched by the same engine as SCAN MATCH, so they carry the same metacharacter trap: ~cache[ab]:* grants cachea:1 and cacheb:1 and denies the literal cache[ab]:1.
vishal-bala
marked this pull request as ready for review
August 25, 2026 14:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What's wrong
BaseCache.clear()built itsSCAN MATCHpattern by interpolating the cache name directly. Per the SCAN docs that pattern is glob-style, and the name is caller-supplied and unvalidated — so a name containing*,?,[or\produced a pattern matching keys the cache does not own, andclear()then deleted them.Reproduced on Redis 8.4.5:
So
SemanticCache(name="cache[ab]").clear()wipes two unrelated caches and leaves its own entries intact. It is silent — no error, and the caller believes their cache was cleared.This is data loss on a destructive path, not a matching quirk: low likelihood (unusual names) but high consequence, with a blast radius covering other keys in the same keyspace. In a multi-tenant naming scheme such as
EmbeddingsCache(name=f"cache:{tenant_id}")a tenant registeringx[ab]destroys other tenants' entries.Grepping
match=across the repo found the same shape in three more places, two of them also destructive:BaseCache.clear/aclearSemanticRouter._route_patterndelete_route_referencesbuild_scan_match_patternsRENAME/DUMP-RESTORE-DELpathMigrationPlanner._sample_keysand the async twinThe fix
A
match_pattern(*segments)builder inredisvl/utils/utils.pyescapes each literal segment and appends the trailing glob. All four sites now build patterns through it;build_scan_match_patternsis itself a chokepoint, so fixing it coversexecutor,async_executor,validationandasync_validationat once.Exposing a builder rather than a bare
escape()is deliberate. Escaping at the call site is a step the next contributor can forget, and no lint rule can express the invariant here: ruff and flake8 are unconfigured, mypy is not strict, and patterns are consumed several call sites away from where they are built. A builder makes the omission unrepresentable — there is no other way to make a pattern.The escape set is
\,*,?,[.],^and-are deliberately absent: they are only meaningful inside a[...]class, which can never open once[is escaped. Confirmed empirically —cache\[ab]:*andcache\[ab\]:*return identical results.Escaping rather than rejecting such names. Rejecting at construction would be a breaking change that orphans keys: a deployment whose cache is already named
cache[ab]could no longer construct the object to clean up after itself. If maintainers prefer validation, it wants a deprecation cycle rather than a straight raise.Testing
Two questions, separated by what can actually answer each:
tests/unit/test_scan_pattern_escaping.pycovers all four sites hermetically, includingaclearand both planners.tests/integration/test_embedcache.pyadds the cache regression (three caches colliding under an unescaped glob; clear one, assert the others survive), andtests/integration/test_migration_v1.pyadds the migration and router prefix case.No glob matcher is reimplemented anywhere, so nothing can drift from Redis. Every new test was checked to fail without the fix.
Verified: mypy clean (119 files), 1421 unit tests pass, integration green across embedcache and migration.
Deliberately out of scope
Four pre-existing issues surfaced while working in this code. None is caused or worsened by this change, and each wants its own PR:
BaseCache.clear()has a containment bug escaping cannot fix._get_prefix()isf"{name}:", so a cache namedfooclearingfoo:*deletes the entries of a cache namedfoo:bar— ordinary names, no metacharacters, same blast radius.SearchIndex.clear()already does this correctly by deleting the doc idsFT.SEARCHreturns;SemanticCacheshould route through it, andEmbeddingsCache(which has no index) should verify each scanned key's prefix segment before deleting.clear()/aclear()never advance aMappingcursor on Redis Cluster, so they re-scan page 1 forever. Notecursor = cursor_intis not the fix — a dict cannot be passed back as a SCAN cursor; per-node iteration viascan_iteris. Already addressed on a separate branch.async_plannerappends the key separator where the other three sites do not, so it samples a narrower key set than the index covers —FT.CREATE PREFIXis a literal string-prefix match (verified:PREFIX 1 zzglobgl[ab]indexes only its own keys), which makes the sync planner the correct one. Flagged in a comment here rather than changed, since fixing it is a behaviour change. The natural follow-up consolidates all three pattern builders and dropsbuild_scan_match_patterns' unusedkey_separatorparameter.delete_route_referencessplits with a hardcoded":"rather thankey_separator, and raises afterdrop_keyshas already deleted.Note
Medium Risk
Changes destructive SCAN/delete and migration enumeration behavior for names with glob metacharacters (fixes incorrect cross-key deletion); otherwise localized to pattern building with broad test coverage.
Overview
Fixes a data-loss bug where caller-supplied cache names, index prefixes, or route names containing Redis glob characters (
*,?,[,\) were interpolated intoSCAN MATCHpatterns unescaped, so destructive paths could delete or touch keys the caller did not own (e.g.cache[ab]clearingcachea/cacheb).Adds
match_pattern(*segments)inredisvl/utils/utils.pyto build escaped literal-prefix patterns and routes all pattern construction through it:BaseCache.clear/aclear,SemanticRouter._route_pattern,build_scan_match_patterns(migration executor/validation), and sync/async migration key sampling.docs/user_guide/installation.mdgains ACL notes (TTL refresh needsEXPIRE+ write key access) and documents that ACL key patterns use the same glob semantics asSCAN MATCH.Unit tests assert emitted patterns; integration tests verify Redis behavior for colliding cache names and migration/router prefixes.
Reviewed by Cursor Bugbot for commit edc99b3. Bugbot is set up for automated code reviews on this repo. Configure here.