Vector search failures are not always compile errors. An application can generate embeddings, store them, create an index, and return ten results while still being structurally wrong. A dimension mismatch may appear only during ingestion. An HNSW migration may fail after a model upgrade. A hybrid ranker may silently favor one retriever because raw scores were added together.
These mistakes share one cause: treating an embedding like an ordinary numeric array. An embedding belongs to a specific model, dimension, distance metric, and indexing strategy. The retrieval pipeline must preserve those assumptions from ingestion through search and later upgrades.
Every embedding model returns a vector with a fixed or configured number of elements. The PostgreSQL column must use the same dimension. A vector(1536) column cannot accept a 768-element local embedding or a 3,072-element cloud embedding.
The safest design stores the embedding model and expected dimension in configuration, validates every generated vector before saving it, and treats a dimension change as a schema and data migration. Matching dimensions are necessary, but they do not make embeddings from different models compatible.
public sealed record SearchOptions(string EmbeddingModel, int EmbeddingDimensions);
static void ValidateEmbedding(ReadOnlyMemory<float> vector, SearchOptions options)
{
if (vector.Length != options.EmbeddingDimensions)
{
throw new InvalidOperationException(
$"Model '{options.EmbeddingModel}' returned {vector.Length} dimensions, " +
$"but the database expects {options.EmbeddingDimensions}.");
}
}
ReadOnlyMemory<float> vector =
await embeddingGenerator.GenerateVectorAsync(text, cancellationToken: ct);
ValidateEmbedding(vector, options);High-dimensional embeddings create a second problem: the column may store them, but the index may not support them. pgvector regular vector indexing supports up to 2,000 dimensions. A 3,072-dimension embedding therefore does not fit the normal HNSW plan.
The usual fixes are to request a reduced dimension when the provider supports it, select a smaller embedding model, use halfvec indexing for up to 4,000 dimensions, or choose another strategy. Confirm the model output and index limit before committing to a schema.
CREATE TABLE "DocumentChunks"
(
"Id" uuid PRIMARY KEY,
"Text" text NOT NULL,
"Embedding" vector(1536) NOT NULL
);
CREATE INDEX "IX_DocumentChunks_Embedding_Hnsw"
ON "DocumentChunks"
USING hnsw ("Embedding" vector_cosine_ops);
-- A 3,072-dimension value cannot use this regular vector HNSW index.
-- Reduce dimensions, use a smaller model, evaluate halfvec,
-- or select another store/index strategy.Hybrid search produces two ranked lists: full-text search and vector similarity. Their raw scores do not share a common scale or meaning. Adding them directly gives the numerically larger scoring system more influence.
Reciprocal Rank Fusion is a safer default because it combines rank positions rather than raw scores. A document receives 1 / (k + rank) from each list in which it appears. Documents ranked highly by both retrievers rise naturally, and a common starting value is k = 60.
public static IReadOnlyList<FusedResult> Fuse(
IReadOnlyList<Guid> keywordIds,
IReadOnlyList<Guid> vectorIds,
int rrfK = 60)
{
var scores = new Dictionary<Guid, double>();
AddRanks(keywordIds, scores, rrfK);
AddRanks(vectorIds, scores, rrfK);
return scores
.OrderByDescending(x => x.Value)
.Select(x => new FusedResult(x.Key, x.Value))
.ToArray();
}
static void AddRanks(IReadOnlyList<Guid> ids, IDictionary<Guid,double> scores, int rrfK)
{
for (int i = 0; i < ids.Count; i++)
scores[ids[i]] = scores.GetValueOrDefault(ids[i]) + 1.0 / (rrfK + i + 1);
}Stored vectors are not generic semantic coordinates. They belong to the model that created them. If queries use a new model while documents still contain old vectors, distance comparisons lose meaning even when both models return the same dimension.
Treat a model upgrade as a data migration. Build a new index version, regenerate the complete corpus in batches, validate retrieval quality, and switch traffic only after the replacement is ready. Store model id, dimension, and index version so mixed data is detectable.
public sealed class DocumentChunk
{
public Guid Id { get; set; }
public required string Text { get; set; }
public required Vector Embedding { get; set; }
public required string EmbeddingModel { get; set; }
public int EmbeddingDimensions { get; set; }
public int EmbeddingIndexVersion { get; set; }
}
var active = db.DocumentChunks.Where(x =>
x.EmbeddingModel == options.EmbeddingModel &&
x.EmbeddingIndexVersion == options.ActiveIndexVersion);HNSW is an approximate nearest-neighbour index. It trades a small amount of recall for much faster search as the corpus grows. That trade-off should be measured, not ignored.
Another mistake is retrieving only the final result count from each leg. If the API returns ten results and each retriever supplies only ten candidates, fusion has little room to find consensus. Retrieve a broader set—often dozens per leg—fuse, and then take the final top results.
int finalTop = request.Top ?? 10;
int candidatesPerLeg = options.CandidatesPerLeg; // for example, 50
Task<IReadOnlyList<RankedCandidate>> keywordTask =
keywordSearch.FindAsync(request.Query, candidatesPerLeg, ct);
Task<IReadOnlyList<RankedCandidate>> vectorTask =
vectorSearch.FindAsync(queryVector, candidatesPerLeg, ct);
await Task.WhenAll(keywordTask, vectorTask);
return ReciprocalRankFusion
.Fuse(keywordTask.Result.Select(x => x.Id).ToArray(),
vectorTask.Result.Select(x => x.Id).ToArray(),
options.RrfK)
.Take(finalTop)
.ToArray();Start with metadata: embedding model, output dimension, column dimension, distance function, and active index version. Then run a small labeled query set in keyword-only, vector-only, and hybrid modes.
A fast HNSW index can still be a poor search system if chunks, dimensions, candidate depth, or fusion logic are wrong. Measure retrieval quality separately from latency.
[ ] Embedding model id is stored and versioned
[ ] Generated vector length matches configuration
[ ] Database vector(N) uses the same dimension
[ ] HNSW supports the selected type and dimension
[ ] Distance operator matches the index operator class
[ ] Keyword and vector raw scores are not added directly
[ ] RRF fuses ranked candidate lists
[ ] Candidate depth exceeds the final top count
[ ] Model changes trigger full re-embedding
[ ] Approximate search is compared with exact search
[ ] Retrieval quality is tested with labeled queriesThe generated vector does not match the dimension declared by the database column, or the selected index cannot support that dimension. Validate vector length before saving and keep the model, dimension, and column definition aligned.
Not with the regular vector type. pgvector indexes vector values up to 2,000 dimensions. Request a smaller embedding when supported, use halfvec indexing up to 4,000 dimensions, or choose another indexing strategy.
The scores use different scales and meanings. Adding them lets one retriever dominate unpredictably. Reciprocal Rank Fusion combines rank positions instead of raw scores.
Yes. Embeddings from different models or versions do not share a reliable vector space. Rebuild the corpus into a new index, validate retrieval, and switch only after the new index is ready.