improvement(search): reduce retrieval overhead

[waleedlatif1]

Summary

• Reduce native OpenAI embedding response size, vector sort overhead, and database admission roundtrips while preserving retrieval results and permission checks.
• Resolve canonical knowledge references without computing document counts.
• Repair the original embedding column constraint for already-supported vector widths. Existing ranking defaults and explicit reranking behavior stay unchanged.

Type of Change

• Performance improvement
• Bug fix

Testing

• 328 paired live searches with reranking off preserved measured relevance and reported token cost, with no search errors. Serial median latency fell 16–19%; the short eight-concurrent-search run improved the median 8%. Tail latency did not consistently improve.
• Includes two 100-question public-corpus runs and a full 80-question fictional enterprise corpus using text-embedding-3-large at a fixed 3072 dimensions. These are regression measurements, not production-scale capacity claims.
• 100 paired recorded-response searches produced identical complete results, scores and costs.
• After merging staging: 408 focused unit tests, 72 PostgreSQL/Redis integration cases, and 3 migration regression tests passed. All canonical migrations applied successfully to a fresh disposable database.
• Real browser query/filter/citation interactions and HTTP MCP authentication, search, document reads, filters and key revocation passed. Repeated the HTTP smoke test after the staging merge.
• Type-check, lint, all 46 repository audits, generated artifact checks, and migration safety passed.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
docs	Ready	Preview	Sep 9, 2026 6:33pm UTC

[greptile-apps]

Greptile Summary

This PR reduces knowledge-search overhead while preserving retrieval semantics and access controls.

• Requests compact base64 Float32 embeddings from native OpenAI and strictly decodes them into number arrays.
• Batches database token-bucket initialization to reduce admission round trips.
• Moves vector ranking into a bounded subquery and retains filtered, globally ordered retrieval.
• Resolves active knowledge-base references without loading aggregate document and token counts.
• Adds the canonical migration needed for all supported embedding widths and broadens migration, integration, and mock coverage.

Confidence Score: 5/5

The PR appears safe to merge with no outstanding correctness, security, migration, or repository-rule issues.

The current changes preserve embedding validation, access-control filtering, ranking behavior, and canonical migration order. The only previous finding was manually resolved after Greptile conceded that the flagged any annotations predated this PR.

Important Files Changed

Filename	Overview
apps/sim/lib/embeddings/providers/openai.ts	Switches native OpenAI embeddings to validated base64 Float32 transport while preserving numeric vectors for callers.
apps/sim/lib/knowledge/search/queries.ts	Bounds vector sorting through a ranked subquery while retaining filtering, permissions, and final ordering.
apps/sim/lib/core/rate-limiter/storage/db-token-bucket.ts	Batches deterministic token-bucket initialization and safely handles empty reservations.
apps/sim/lib/knowledge/application/contexts.ts	Uses a lightweight active knowledge-base reference for child-resource context resolution.
apps/sim/lib/knowledge/service.ts	Adds lightweight canonical knowledge-reference loading without aggregate count queries.
packages/db/migrations/0335_embedding_width_nullable.sql	Removes the legacy 1536-vector NOT NULL restriction while retaining the validated supported-width constraint.
packages/testing/src/mocks/database.mock.ts	Extends the database mock to support the added ranked-subquery behavior without introducing the previously alleged typing regression.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Q[Search request] --> C[Resolve active knowledge reference]
  C --> A[Resolve access scope]
  A --> E[Request OpenAI base64 embedding]
  E --> D[Validate and decode Float32 vector]
  D --> R[Rank eligible embeddings in bounded subquery]
  R --> F[Apply source, tag, and ACL filters]
  F --> O[Return globally ordered top-K results]

Loading

_{Reviews (6): Last reviewed commit: "chore(search): sync staging and regenera..." | Re-trigger Greptile}

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[waleedlatif1]

@greptile

[waleedlatif1]

@greptileai review this PR

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[waleedlatif1]

The PR scope is now performance-only. Please evaluate the final diff against the staging base, rather than the intermediate ranking implementation that was removed.

Both apps/sim/lib/knowledge/reranker.ts and apps/sim/lib/core/rate-limiter/provider-admission.ts are identical to staging. The malformed-response behavior and fixed rerank burst described in the summary therefore remain existing opt-in behavior; this PR does not introduce them. Organization default reranking, title-aware inputs, billing changes and rerank capacity changes were deliberately removed. Their associated hardening can be reviewed with that separate work.

The final change retains embedding transport, narrower vector selection, lightweight canonical references, batched bucket initialization and the independent vector-column constraint repair. New tests assert that ordinary organization and workspace searches do not rerank and that existing explicit opt-in behavior remains available. Please reassess the net diff with that scope.

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

[waleedlatif1]

@greptile

[waleedlatif1]

@cubic-dev-ai review this PR

[cubic-dev-ai]

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.