FIX #2562: send "dimensions" to Nvidia embedding NIM for variable-dimension models - #2561
Open
erichare wants to merge 5 commits into
Open
FIX #2562: send "dimensions" to Nvidia embedding NIM for variable-dimension models#2561erichare wants to merge 5 commits into
erichare wants to merge 5 commits into
Conversation
… models NvidiaEmbeddingRequest never carried `dimensions`, so for nvidia/llama-3.2-nv-embedqa-1b-v2 (384/512/768/1024/2048, default 2048) the NIM always returned 2048-dim vectors and vectorize failed with "did not return expected embedding length. Expect: '512'. Actual: '2048'". - add `dimensions` to NvidiaEmbeddingRequest, omitted when 0 (NIM rejects 0) - acceptsNvidiaDimensions(modelConfig): model has no fixed vector-dimension; fixed-dimension models (NV-Embed-QA, nv-embedqa-e5-v5) are unchanged - NvidiaEmbeddingClientTest (WireMock) covers both cases
Contributor
📈 Unit Test Coverage Delta vs Main Branch
|
Contributor
Unit Test Coverage Report
|
erichare
changed the title
erichare
marked this pull request as ready for review
August 21, 2026 17:50
Contributor
📉 Integration Test Coverage Delta vs Main Branch (dse69-it)
|
Contributor
Integration Test Coverage Report (dse69-it)
|
Contributor
📉 Integration Test Coverage Delta vs Main Branch (hcd-it)
|
Contributor
Integration Test Coverage Report (hcd-it)
|
Contributor
There was a problem hiding this comment.
The fix is not generalized enough - it can only fix nvidia's problem but not others. I think the key problem is this line https://github.com/stargate/data-api/blob/main/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/VectorizeConfigValidator.java#L355 and specifically model.vectorDimension().get() != 0
it should't go inside this if block and the range dimension should be handled here: https://github.com/stargate/data-api/blob/main/src/main/java/io/stargate/sgv2/jsonapi/service/resolver/VectorizeConfigValidator.java#L372 and add some tests here: https://github.com/stargate/data-api/blob/main/src/test/java/io/stargate/sgv2/jsonapi/api/v1/CreateCollectionIntegrationTest.java#L734
Hazel-Datastax
approved these changes
Aug 21, 2026
| protected static boolean acceptsNvidiaDimensions( | ||
| EmbeddingProvidersConfig.EmbeddingProviderConfig.ModelConfig modelConfig) { | ||
| return modelConfig.parameters().stream() | ||
| .anyMatch(parameter -> parameter.name().equals("vectorDimension")); |
Contributor
There was a problem hiding this comment.
Could we explain in java doc why parameter -> parameter.name().equals("vectorDimension") could decide whether models accept dimensions or not
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
NvidiaEmbeddingRequestnever carrieddimensions, so the Collection/Table dimension never reached the NIM. For variable-dimension models (nvidia/llama-3.2-nv-embedqa-1b-v2: 384/512/768/1024/2048, default 2048) the NIM always returned 2048-dim vectors and vectorize failed:Changes
NvidiaEmbeddingRequest.dimensions, serialized only when non-zero. The NIM rejectsdimensions: 0with HTTP 400, so it must be omitted (not zeroed) for fixed-dimension models.Verification
dimensions→ 2048;512→ 512;384→ 384;0→ HTTP 400../mvnw test -Dtest=NvidiaEmbeddingClientTest,EmbeddingProviderErrorMessageTest,EmbeddingGatewayClientTest,OpenAiEmbeddingClientTest→ 19 tests, 0 failures.