A single prompt is stateless. The model reads what you sent, produces a response, and remembers nothing on its own. Yet the conversations people build feel like they remember: a support assistant recalls the order number you gave three turns ago, a booking flow knows you already picked a date. That memory is an illusion you construct, and the craft of constructing it well is dialogue state management.
Dialogue state is the set of facts a conversation needs to carry forward to stay coherent. The user's name, the choices they have made, what step of a process they are on, what the system is still waiting to learn. Because the model does not retain any of this between calls, you have to capture it, store it, and feed the relevant parts back into each prompt. Do it carelessly and the conversation forgets things, contradicts itself, or asks for information it was already given.
This guide is the definitive overview for someone serious about getting multi-turn conversations right. It covers what state is, how to represent it, how to update it as the dialogue moves, and how to feed it back without bloating the prompt. If you are brand new to the idea, How Conversations Remember: State Inside Your Prompts is a gentler starting point.
What Dialogue State Actually Is
State is everything the next turn needs that did not just arrive in the latest message.
The Components of State
- Established facts: information the user has provided that remains true
- Decisions made: choices the conversation has locked in
- Process position: where the user is in a multi-step flow
- Open obligations: what the system still needs to collect or do
Naming these components matters because lumping them together leads to a vague blob of history that is hard to update and easy to corrupt. Treating state as structured data, rather than raw transcript, is the central move.
Representing State Explicitly
The naive approach is to send the entire conversation history every turn and hope the model extracts what it needs. It works until it does not.
Why Raw History Falls Short
As a conversation grows, the full transcript becomes long, expensive, and noisy. The model has to rediscover the relevant facts on every turn, and important details can get lost among the chatter. Worse, contradictions in the history confuse the model about what is currently true.
A Structured State Object
A better approach maintains an explicit state object alongside the conversation: a compact record of the established facts, decisions, position, and open items. You update this object as the dialogue progresses and inject a summary of it into each prompt. The full transcript becomes optional context; the state object becomes the source of truth. We walk through building one in Wiring Memory Into a Multi-Turn AI Conversation.
Updating State as the Dialogue Moves
State is not static. Each turn can add facts, change decisions, or advance the process.
Deciding What Changes
After each user message, you determine what new information arrived and how it modifies the state. Did the user provide a value you were waiting for? Change an earlier choice? Move to a new step? The update logic can be code you write, or you can ask the model itself to propose an updated state, which you then validate.
Handling Corrections
Users change their minds. A robust state representation makes the current value the single source of truth, so when someone says "actually, make it Thursday," the state reflects Thursday and the old value does not linger to cause contradictions later.
Feeding State Back Into the Prompt
Maintaining state is only useful if the relevant parts reach the model on each turn.
Injecting a State Summary
Rather than dumping the whole state object, include a concise summary of what the model needs for the current turn. If the conversation is collecting shipping details, the prompt foregrounds the address fields collected and still missing, not the entire history. Keeping the injected state focused keeps the prompt small and the model's attention on what matters.
Balancing Completeness and Size
Too little state and the model forgets; too much and the prompt bloats and the model loses focus. Finding the right summary for each turn is a judgment call, and getting it wrong is a frequent source of trouble, as we detail in Seven Ways Conversational Prompts Lose Their Thread.
Handling Long Conversations
Eventually a conversation outgrows any prompt window. State management is how you cope.
Summarization and Pruning
- Periodically summarize older turns into compact facts and keep the summary, not the raw turns
- Prune details that are no longer relevant once a process step is complete
- Preserve the structured state object even when you discard transcript
Because the state object holds the durable facts, you can trim the conversation history aggressively without losing what the dialogue depends on.
Validating and Recovering
Things go wrong: the model proposes a bad state update, a user gives contradictory information, a value goes missing.
Building in Checks
Validate state updates before committing them. Confirm a value with the user when it is ambiguous or high-stakes. Keep the ability to recover gracefully when the state and the conversation diverge. These safeguards separate a demo that works once from a system that works in production. The opinionated version of these safeguards lives in Holding a Conversation Together Across Many Turns.
Dividing Work Between Code and Model
A recurring design question is who should own each part of state management: your deterministic code or the model itself. The answer is usually both, with a clear division of responsibility.
What the Model Does Well
The model excels at reading messy, natural human input and pulling structured facts out of it. When a user says "let's do sometime late next week, maybe Thursday," turning that into a concrete date is exactly the kind of fuzzy interpretation the model handles gracefully and code handles poorly.
What Code Should Own
Code should own anything that must be reliable: validating a proposed value, deciding what gets committed to state, computing which required fields remain open, and enforcing the rules of the flow. These are deterministic decisions, and a model's small probability of error makes it the wrong tool for them.
- Let the model propose updates from messy input
- Let code validate and commit those updates
- Keep the flow's control logic in code, not in the model's discretion
This division, model for interpretation and code for control, is one of the most reliable architectural patterns in conversational systems, and it scales from a simple booking bot to a complex multi-step assistant.
Designing State for a Specific Task
State is not generic; it is shaped by what the conversation is trying to accomplish.
Start From the Goal
Before designing a state object, articulate what a completed conversation looks like. A booking needs a service, date, time, and contact. A troubleshooting flow needs the symptom, the steps already tried, and the current hypothesis. Listing the fields a successful outcome requires defines your state schema directly, and a schema derived from the goal rarely carries dead weight.
Keep It Minimal
Resist the urge to track everything a user might say. State should hold what the conversation needs to act on, not a complete record of the exchange. A lean state object is easier to update, validate, and inject, and it keeps the whole system comprehensible as the conversation grows.
Frequently Asked Questions
Is dialogue state management the same as just sending chat history?
No. Sending history is the naive version, where the model rediscovers relevant facts every turn. State management maintains an explicit, structured record of the facts that matter, updates it deliberately, and injects a focused summary. The structured object is what makes the difference at scale.
Where do I store the state object?
In whatever your application already uses: a session store, a database record, or an in-memory object tied to the conversation. The store is an implementation detail. What matters is that the object persists between turns and is updated reliably as the dialogue progresses.
Should the model or my code manage the state?
Often both. Code handles deterministic updates and validation; the model can help extract facts from messy user input and propose updates. The safe pattern is to let the model suggest and your code validate before committing, so a model error does not silently corrupt the state.
How much state should I put in each prompt?
Only what the current turn needs, summarized compactly. Include the facts, decisions, and open items relevant to where the conversation is now. Dumping the full state every turn bloats the prompt and dilutes the model's attention.
What happens when the conversation gets too long?
You summarize older turns into compact facts and prune transcript you no longer need, while preserving the structured state object. Because the durable facts live in the object rather than the raw history, aggressive trimming is safe.
How do I handle a user contradicting earlier information?
Make the current value in the state the single source of truth and overwrite the old one on a correction. A representation where the latest value wins prevents stale data from lingering and contradicting the model on later turns.
Key Takeaways
- The model is stateless; the memory in a conversation is something you construct and maintain
- Treat state as a structured object of facts, decisions, position, and open items, not raw transcript
- Update the state deliberately each turn and let the latest value be the single source of truth
- Inject a focused summary of relevant state, not the whole object, to keep prompts small and attention sharp
- Summarize and prune long conversations while preserving the durable state object, and validate updates before committing