Embeddings and vector search sit at the core of most modern AI applications—retrieval-augmented generation, semantic search, recommendation engines, duplicate detection. But most teams that build with them measure almost nothing beyond "it seems to return relevant results." That instinct is expensive. When retrieval breaks, the whole system breaks: your RAG pipeline hallucinates, your search returns stale results, your recommendations drift. The failure is invisible until a user complains or you run an audit and find the rot.
Getting measurement right means defining what "good retrieval" actually looks like for your use case, instrumenting the pipeline to capture signal continuously, and reading the metrics with enough nuance to know when to intervene. This article does all three. It covers the specific KPIs that matter for embeddings and vector search, how to compute and log them, and how to interpret the signals when something goes wrong.
If you're newer to how the underlying technology works, A Framework for How Generative AI Works gives useful scaffolding before you dive into measurement. If you're already instrumenting a broader AI stack, this article fits naturally alongside How to Measure How Generative AI Works: Metrics That Matter.
Why Embeddings and Vector Search Fail Silently
Most vector search failures don't throw exceptions. The system returns results—they're just wrong. Understanding the failure modes shapes the measurement strategy.
The Three Root Causes of Bad Retrieval
Embedding quality degradation. The model that generated your stored vectors may no longer match the model generating query vectors. This happens when you upgrade your embedding model mid-deployment without re-indexing, or when the domain of incoming queries drifts away from the training distribution of your model.
Index configuration drift. Vector indexes (HNSW, IVF, flat) have tunable parameters—ef_search in HNSW, nprobe in IVF—that trade recall for latency. These defaults are rarely optimal for your data size and query patterns. As your corpus grows, parameters that were fine at 100K vectors may silently degrade at 10M.
Corpus staleness. If your documents are updated but your embeddings aren't re-generated, the vectors represent old content. The index looks healthy; the content is wrong.
Each failure mode requires a different measurement response, which is why a single "did it return results?" metric misses almost everything.
The Core Retrieval Quality Metrics
These are the foundational KPIs for embeddings and vector search metrics. You need at least two or three of these running in production before you ship anything customer-facing.
Recall@K
Recall@K answers: of all the truly relevant documents for this query, what fraction appeared in the top K results?
- Typical targets: Recall@5 > 0.80 for high-stakes use cases (legal, medical, customer support); Recall@10 > 0.70 is a reasonable starting bar for general search.
- How to compute it: You need labeled ground truth—query-document relevance pairs. Build a golden evaluation set of 200–500 representative queries with known relevant documents. Run the retriever, measure overlap with ground truth.
- The trap: Teams optimize Recall@10 while shipping Recall@3 to users. Measure at the K you actually use in production.
Precision@K
Precision@K answers: of the K results returned, what fraction are actually relevant?
High recall, low precision means you're retrieving the right documents but drowning them in noise. In a RAG pipeline, sending 10 retrieved chunks to the LLM where only 2 are relevant inflates cost and increases hallucination risk—the model has more bad context to reason from.
- Practical floor: Precision@5 > 0.60 is worth defending. Below 0.50 and you're essentially sending coin-flip context to your LLM.
Mean Reciprocal Rank (MRR)
MRR weights results by position. If the first truly relevant document appears at rank 1, the reciprocal rank is 1.0. If it appears at rank 5, it's 0.2. MRR is the mean of those reciprocal ranks across queries.
MRR is especially useful when your UI shows ranked results and users click the first plausible link. It captures whether the best result is near the top, not just whether it appeared somewhere in the top K.
Normalized Discounted Cumulative Gain (NDCG)
NDCG extends MRR to handle graded relevance (not just binary relevant/not-relevant). If you score documents as highly relevant, somewhat relevant, or irrelevant, NDCG captures the difference between returning highly relevant docs at rank 1 versus rank 4.
Use NDCG when your domain has meaningful relevance gradations. For most keyword-replacement use cases, Recall@K and Precision@K are sufficient. For nuanced recommendation systems, NDCG earns its complexity.
Latency and Throughput: The Operational KPIs
Retrieval quality metrics tell you what you're returning. Latency tells you whether anyone will wait long enough to receive it.
P50, P95, P99 Latency
Mean latency hides the tail. A search system with 40ms mean latency and 2,000ms P99 is broken for roughly 1 in 100 users.
- Target ranges: P95 < 100ms for interactive search; P95 < 300ms for RAG pipelines where the LLM adds another 1–3 seconds anyway.
- Instrument separately: Log embedding generation latency and index query latency as separate spans. When P95 degrades, you need to know which component caused it.
Queries Per Second (QPS) at Target Latency
Peak QPS at your P95 target is your actual capacity ceiling. Test this during index configuration, not after a traffic spike.
Index Build and Re-index Time
Often ignored until it matters: how long does it take to rebuild the index when you upgrade your embedding model or add 500K new documents? If re-indexing takes 18 hours and your corpus updates daily, you have a staleness problem baked into your architecture.
Embedding Quality Metrics
These metrics assess the representations themselves, independent of retrieval outcomes.
Cosine Similarity Distribution
Run a random sample of 1,000 query-document pairs (both relevant and non-relevant) and plot the cosine similarity distribution. A well-calibrated embedding model shows:
- High similarity (> 0.80) for known relevant pairs
- Low similarity (< 0.40) for confirmed non-relevant pairs
- Clear separation between the two distributions
When the distributions overlap heavily, your embedding model isn't creating a useful semantic space for your domain. This is a signal to fine-tune or switch models.
Intra-cluster Cohesion
For recommendation or deduplication use cases, cluster your embeddings (k-means works fine as a diagnostic tool) and measure average intra-cluster cosine similarity. Healthy embeddings of topically similar content typically cluster with intra-cluster similarity > 0.75. If similar documents scatter across clusters, the embedding space isn't reflecting your domain's semantic structure.
Query-Corpus Distribution Shift
Compare the embedding distribution of queries to the embedding distribution of your corpus using Maximum Mean Discrepancy (MMD) or, more practically, by checking whether average query-to-nearest-corpus-document similarity is declining over time. A declining trend signals that user queries are drifting into territory your corpus doesn't cover.
Building a Golden Evaluation Set
Every metric above is meaningless without labeled data. Building the golden set is the highest-leverage work you can do before you optimize anything.
Practical Construction Steps
- Sample representatively. Pull 300–500 real queries from logs. If you don't have logs yet, write queries in the voice of your actual users—this is not where to use hypothetical edge cases.
- Label at least 3 documents per query. For each query, identify the top relevant documents in your corpus manually. Two-reviewer agreement with a tiebreaker handles ambiguous cases.
- Include hard negatives. For each query, also label 2–3 documents that seem topically related but are genuinely not relevant. These stress-test Precision@K.
- Version control the set. As your corpus evolves, update the golden set quarterly. Frozen evaluation sets become misleading as documents change.
- Automate evaluation against it. Run your retriever against the golden set on every model or index configuration change. Treat a Recall@5 drop of more than 3 percentage points as a blocking regression.
Instrumenting the Pipeline
Defining metrics is the easy part. Capturing them continuously in production requires deliberate instrumentation.
What to Log at Query Time
For every query, emit a structured log record with:
- Query text (or a hash if PII concerns apply)
- Embedding generation latency (ms)
- Index query latency (ms)
- Top-K document IDs returned
- Top-K similarity scores
- Total end-to-end retrieval latency
Implicit Feedback Signals
When you have user interaction data, implicit signals supplement offline evaluation:
- Click-through rate on result rank. If users consistently click the third result rather than the first, MRR and your ranking logic deserve scrutiny.
- Query reformulation rate. If users immediately rephrase after seeing results, retrieval likely failed. Track this as a proxy for Precision failures.
- Session abandonment after search. High abandonment without a click is the strongest implicit signal that the result set wasn't useful.
These signals don't replace labeled evaluation, but they catch distribution drift faster than quarterly audits. The tradeoffs between offline and online evaluation approaches connect to broader AI system design principles covered in How Generative AI Works: Trade-offs, Options, and How to Decide.
Alerting Thresholds
Set alerts on:
- Recall@5 dropping more than 5 points from your established baseline
- P95 retrieval latency exceeding 2× your established baseline
- Average top-1 similarity score declining by more than 0.05 over a 7-day rolling window (signals corpus or query drift)
Reading the Signal: Diagnosis Patterns
When metrics move, the pattern of movement tells you which lever to pull.
| Symptom | Likely Cause | First Intervention | | ------------------------------ | --------------------------------------------------------------------- | ------------------------------------------------------------- | | Recall drops, Precision stable | Index recall issue (nprobe/ef_search too low) | Increase search parameters; benchmark latency cost | | Precision drops, Recall stable | Embedding model-corpus mismatch or hard queries entering distribution | Inspect low-precision queries; consider domain fine-tuning | | Both drop | Embedding model version mismatch or corpus staleness | Check model versions; audit embedding timestamps | | Latency spikes, quality stable | Index size growth hitting memory limits | Scale index sharding or move to approximate search | | Similarity scores collapse | Query distribution shift | Expand corpus or fine-tune embedding model on new query types |
These patterns connect directly to the tool selection decisions you'll face when building retrieval infrastructure—The Best Tools for How Generative AI Works covers the vector database and embedding provider landscape if you're evaluating options.
Frequently Asked Questions
What's the minimum viable measurement setup for a new RAG application?
Start with a golden evaluation set of 200 queries, automated Recall@5 and Precision@5 against it, and P95 latency logging in production. That's enough to catch the most common failure modes without over-engineering before you have real user behavior to learn from.
How often should I re-evaluate against my golden set?
Run it on every material change: embedding model updates, index parameter changes, major corpus additions. Outside of changes, monthly automated evaluation catches slow drift. Quarterly is the absolute minimum for any production system.
Should I use cosine similarity or dot product for my similarity metric?
For normalized embeddings (which most modern models produce), cosine similarity and dot product are mathematically equivalent. For unnormalized embeddings, cosine similarity is more reliable because it removes magnitude differences. Check your embedding model's documentation—most specify which metric they were trained to optimize.
How many labeled examples do I need for meaningful Recall@K estimates?
200–300 queries gives you enough statistical power to detect meaningful differences (3–5 percentage point changes in Recall@K). Fewer than 100 queries produces estimates with confidence intervals too wide to trust for engineering decisions. More than 1,000 is usually unnecessary unless you have many distinct query types that need separate evaluation.
Can I use an LLM to auto-label my golden evaluation set?
Yes, as a starting point—but treat LLM-generated labels as a draft requiring human review. LLMs miss domain-specific relevance nuances and can be overconfident about borderline cases. A hybrid approach (LLM labels first, human reviewers validate and correct) cuts labeling time by 60–70% while maintaining label quality. Understanding the limitations of AI-generated outputs is part of the broader literacy discussed in The How Generative AI Works Checklist for 2026.
What's the difference between embedding metrics and retrieval metrics?
Embedding metrics (cosine similarity distribution, intra-cluster cohesion) measure the quality of the representations themselves—how well the model has encoded semantic meaning. Retrieval metrics (Recall@K, MRR) measure whether the search system finds the right documents given those representations. You can have high-quality embeddings and still have poor retrieval if your index configuration is wrong, and vice versa. Both layers need independent measurement.
Key Takeaways
- Recall@K and Precision@K are the non-negotiable starting metrics—measure at the exact K you ship to users, not a more flattering number.
- A golden evaluation set of 200–500 labeled queries is prerequisite infrastructure, not a nice-to-have. Build it before you optimize anything.
- Log embedding latency and index query latency as separate spans—when P95 degrades, you need to know which component failed.
- Cosine similarity distribution between relevant and non-relevant pairs is the fastest diagnostic for embedding model fitness. Heavy overlap means the model isn't right for your domain.
- Implicit feedback signals (click-through rate, query reformulation) catch distribution drift faster than quarterly audits—instrument them from day one.
- The pattern of metric movement—not just the direction—tells you which layer is broken: embedding quality, index configuration, or corpus freshness each leave distinct signatures.
- Re-index time and corpus staleness are operational risks most teams ignore until they become incidents. Measure them before you need to.