A checklist earns its place only if you actually run it. The list below is built to be used, not admired, before you ship a grounded system and again whenever the answers start drifting. Each item carries a one-line reason, because a checklist whose items you do not understand is a checklist you will skip under deadline pressure. Work through it top to bottom; the order roughly matches the order in which problems tend to bite.
This is a 2026 checklist in the sense that it reflects what reliably works now: lean context, auditable retrieval, honest instructions, and standing evaluation. The fundamentals have settled, and these are them.
Print it, paste it into your project tracker, or keep it open in a tab. The value is in the doing. Treat any item you are tempted to skip as the one most likely to bite you, because the items teams skip are precisely the ones whose absence is hardest to notice until an answer goes wrong in front of a user.
Source Material
Before You Index Anything
- Confirm the documents actually contain answers to the questions users will ask. If the answer is not in your corpus, no amount of retrieval finds it.
- Remove outdated and superseded documents. Stale sources surface in retrieval and produce wrong answers that look authoritative.
- Strip boilerplate, navigation, and broken formatting from each document. Clean input is the foundation of clean retrieval.
Document Coverage
- Map your expected questions to specific documents. Gaps in coverage show up as confident fabrications later.
- Note where documents contradict each other so you can prune or reconcile before they confuse the model.
Chunking
Splitting Strategy
- Split on natural boundaries, paragraphs and sections, not raw character counts. Mechanical splits cut ideas in half and ruin retrieval.
- Add a small overlap between adjacent chunks so a concept that straddles a boundary survives in at least one piece.
- Match chunk size to your question type: small chunks for factual lookups, larger ones for synthesis. The reasoning is detailed in Grounding Prompts with Retrieved Context: Best Practices That Actually Work.
Retrieval
Before Trusting the Model
- Inspect the retrieved chunks for a sample of real questions before reading any model output. If a human cannot answer from them, neither can the model.
- Start with three to five retrieved chunks and adjust based on whether answers are missing detail or wandering.
- Log which chunks were retrieved for each question. The log tells you instantly whether a failure was in finding facts or using them.
Query Quality
- Consider rewriting vague user questions into clearer search queries before retrieval. Better queries return better chunks.
- Test retrieval against paraphrased versions of the same question to confirm it is matching meaning, not just exact words.
Prompt Construction
The Instruction
- Tell the model to answer only from the supplied context. Without this, it blends in training knowledge and disguises guesses as facts.
- Grant explicit permission to say the answer is not present. This single sentence prevents a large class of fabrications.
- Require a citation to the specific chunk behind each claim. Citations expose fabrication and make answers verifiable.
Layout
- Mark the context block clearly so the model never confuses it with the instruction or the question.
- Place the strongest passage where the model weights it most, near the start or end, not buried in the middle.
Handling Conflict and Uncertainty
When Sources Disagree
- Instruct the model to flag conflicting sources rather than silently choosing one. A surfaced conflict is fixable; a hidden one is a wrong answer.
- Decide in advance how recency breaks ties, and prune the loser from your index where you can.
Evaluation
Before and After Every Change
- Maintain a standing set of ten to twenty real questions with known answers and run it after every change. This catches regressions before users do.
- Change one variable at a time, chunk size, retrieval count, or prompt wording, so you can attribute each result. The full sequence appears in Build a Grounded Prompt Pipeline in Eight Concrete Steps.
- Re-run evaluation on a schedule even when nothing changed, since document updates shift behavior over time.
Cost and Latency
Keeping the System Affordable
- Track tokens spent per answer and watch for creep as your context grows. Lean context is not just better for quality; it is cheaper and faster.
- Cache retrieval results for repeated questions where your content allows it, so common queries do not re-run the full pipeline.
- Set a ceiling on retrieved passages and prompt length so a single oversized question cannot blow your budget or your response time.
Watching Response Time
- Measure end-to-end latency, not just the model call, since retrieval and assembly add up. Users feel the total, not the parts.
- Decide what latency your use case can tolerate before optimizing, so you do not over-engineer speed the user will never notice.
Ongoing Operation
Keeping It Healthy
- Re-index documents whenever they change so answers stay current without retraining.
- Watch the rate of refusals and wrong answers in production and treat sudden shifts as a signal to inspect retrieval. The failure modes to watch for are cataloged in 7 Common Mistakes with Grounding Prompts with Retrieved Context.
- Keep a sample of real production questions flowing back into your test set so it reflects how people actually ask, not how you imagined they would.
- Review refusals periodically; a cluster of refusals often points to a coverage gap in your source material rather than a model problem.
Security and Access
Before Exposing the System
- Confirm that retrieval only surfaces documents the asking user is allowed to see. A grounded system can leak confidential material if it indexes everything and filters nothing.
- Strip secrets, credentials, and personal data from documents before indexing, since anything in a chunk can appear in an answer.
- Decide whether your chosen tools send text to third parties, and confirm that is acceptable for the sensitivity of your content.
After Launch
- Audit a sample of answers periodically for accidental disclosure, treating a single leak as a serious incident rather than a tuning issue.
- Keep access rules and the index in sync, so a document removed for permission reasons stops being retrievable immediately.
Frequently Asked Questions
Do I need to run the whole checklist every time?
Run the full list before launch and after major changes. For small tweaks, the retrieval, prompt, and evaluation sections are the ones that catch most problems and are worth a quick pass each time.
Which section is most often skipped?
Retrieval inspection. Teams jump straight to prompt wording without checking whether the right chunks were returned, then wonder why nothing improves. Never skip looking at retrieval.
Why include justifications instead of just the items?
Because an item you understand is one you will keep doing. Justifications turn the checklist from a ritual into a tool you can adapt when your situation differs from the default.
How does this checklist change for synthesis tasks?
Increase retrieval breadth so chunks come from multiple sources, and add an instruction to compare and reconcile them. The rest of the checklist applies unchanged.
How do I keep the checklist current as models change?
Treat it as a living document, not a fixed form. Each time a model upgrade or a retrieval change shifts your results, note what the new behavior was and which checklist item caught it or missed it. When an item stops earning its place because the model now handles that case on its own, retire it; when a new failure mode appears in production, add a check that would have surfaced it before launch. Reviewing the list quarterly against your last batch of incidents keeps it honest and prevents it from drifting into a ritual that no longer matches how your stack actually behaves.
Key Takeaways
- Verify your source material contains the answers and is current before indexing anything.
- Chunk on natural boundaries with slight overlap, and size chunks to match the kind of question.
- Inspect and log retrieval before trusting model output; the right chunks are the precondition for right answers.
- Instruct the model to stay in context, admit uncertainty, and cite sources on every claim.
- Run a standing test set after every change and re-index documents as they evolve.