There is a lot of folklore around prompting models through multi-step decisions, and much of it is repeated with total confidence by people who have done it twice. Some of the advice is harmless. Some of it actively makes results worse, because it encourages habits that look sophisticated and produce fragile chains that fail in exactly the ways nobody warned you about.
Sequential decision making sits in a strange spot. It is powerful enough to feel like magic when it works, which means people build elaborate mental models to explain why it works, and those models are often wrong. The gap between the story people tell about the technique and what is actually happening leads to predictable mistakes.
This article takes the most common claims, holds them up against how the practice behaves in real use, and replaces the myth with the accurate picture. The goal is not to be contrarian. It is to strip away the beliefs that quietly sabotage results.
The Myth That More Steps Means Better Reasoning
The single most persistent belief is that breaking a decision into more steps always improves the quality of reasoning. It does not.
What People Believe
The intuition is that decomposition is free. If two steps are good, ten steps must be better, because you are forcing the model to think harder about each piece. People build sprawling chains under the assumption that granularity equals rigor.
What Actually Happens
Every additional step is another opportunity for error, drift, and lost context. Past a certain point, more steps make a chain less reliable, not more, because the failure surface grows faster than the benefit. The right number of steps is the smallest number that captures the genuinely distinct decisions in the problem. Decomposition is a tool, not a virtue, and over-applying it is one of the most common ways competent people produce brittle results.
The Myth That the Model Remembers Everything
People assume that once a constraint is stated in a chain, the model holds it for the rest of the sequence. This assumption causes a large share of real-world failures.
What People Believe
The mental model is that context is a perfect, durable memory. State a rule at step one and it governs every step afterward, automatically and reliably.
What Actually Happens
Constraints fade as a chain gets longer. A rule stated clearly early on gets diluted by everything that comes after it, and by the later steps the model is no longer reliably honoring it. The accurate practice is to restate governing constraints at each major decision point rather than trusting them to persist. Treating context as durable memory rather than a fading signal is a direct cause of state drift.
The Myth That Step-by-Step Reasoning Is an Audit Trail
When a model explains its reasoning at each step, it looks like you can audit the decision. This appearance is misleading.
What People Believe
The belief is that visible reasoning equals verifiable reasoning. If the model showed its work, you can trust the work, and you have a record you could defend later.
What Actually Happens
The explanations are generated text, produced alongside the answer, and they may not reflect the actual basis for the decision. A chain can show plausible reasoning for a conclusion it reached for entirely different reasons. The reasoning is useful for understanding and debugging, but it is not a guarantee of correctness and should never be treated as a substitute for independently verifying the outcome.
The Myth That a Good Chain Is Universal
Teams often believe that a decision chain that works for one case will work for all similar cases. This belief leads to brittle reuse.
What People Believe
The assumption is that a well-built chain is robust by default. Once it works, you can point it at any similar input and trust the result.
What Actually Happens
Chains are often tuned, implicitly, to the examples they were built against. Subtle features of those examples get baked into the phrasing, and inputs that differ in ways the author never anticipated produce silently wrong results. Real robustness comes from testing a chain against deliberately varied inputs, not from the fact that it worked the first time. Hidden coupling between steps makes this worse, because changing the input can change behavior several steps downstream.
The Myth That Better Prompting Eliminates the Need for Review
Some people believe that with enough prompt-crafting skill, you can build chains reliable enough to run unsupervised. This is the most expensive myth.
What People Believe
The belief is that review is a sign of an immature prompt. Get good enough and you can trust the chain to run end to end without a human checking it.
What Actually Happens
No amount of prompting skill removes the compounding-error problem. The most reliable chains in serious use have explicit checkpoints where a human or a verification step inspects intermediate results. Skilled prompting reduces how often you need to intervene; it does not let you stop intervening. The mature practice pairs good prompts with deliberate review, not one in place of the other.
The Myth That Sequential Prompting Is Only for Hard Problems
A subtler belief holds that chaining decisions is reserved for genuinely complex, high-stakes problems, and that simple work should always use a single prompt. This gets the trade-off backward.
What People Believe
The framing is that complexity justifies the chain. If a problem is hard enough, you decompose it; if it is easy, you do not. People treat the technique as something you reach for only when a single prompt visibly fails.
What Actually Happens
The right question is not how hard the problem is but whether it contains genuinely dependent decisions, where a choice at one stage changes what later stages should do. A simple problem with real dependency benefits from a short chain; a hard problem with no real dependency does not. Plenty of complex tasks are best handled by a single well-structured prompt, and plenty of modest ones genuinely need staging. Dependency, not difficulty, is the signal. Reaching for chains based on perceived difficulty leads people to over-engineer hard-looking problems that had no dependency at all.
The Myth That a Failing Chain Means You Prompted It Wrong
When a chain produces a bad result, the reflex is to assume the prompt was poorly written and to keep rewording it. Sometimes that is right, but often it is not.
What People Believe
The assumption is that any failure is a prompting failure, fixable with better wording. People iterate endlessly on phrasing, convinced the next revision will fix a chain that keeps going wrong.
What Actually Happens
Many chain failures are structural, not verbal. The chain may have too many steps, the wrong checkpoints, or a step that depends on information it does not have. Rewording will not fix a structural problem; restructuring will. Before polishing language for the tenth time, ask whether the chain's shape is wrong: whether a step should be split, merged, removed, or given a verification point. Treating every failure as a wording problem keeps people stuck on chains that need to be rebuilt.
Frequently Asked Questions
Is decomposition ever the wrong approach?
Decomposition itself is sound; over-decomposition is the problem. Break a decision into its genuinely distinct stages and stop there. Adding steps beyond that point increases the failure surface without improving reasoning.
If I cannot trust the model's stated reasoning, why generate it?
Because it is valuable for understanding and debugging, even though it is not a proof of correctness. Use the reasoning to see how the chain is behaving and where it might be going wrong, then verify the actual outcome separately.
Does restating constraints really matter that much?
Yes. State drift is one of the most common silent failures in long chains. Re-stating governing rules at each major step is cheap and is one of the most effective things you can do to keep a chain honest.
Can a well-built chain be reused safely?
It can, but only after you test it against inputs that differ meaningfully from the ones it was built on. Reuse without that testing is where chains quietly break, because they were tuned to their original examples more than their authors realized.
Will better models make these myths obsolete?
Some will soften as models improve, but the structural ones, compounding error and the limits of self-reported reasoning, are properties of chaining decisions, not of any particular model. The accurate practices here are durable.
Is unsupervised chaining ever appropriate?
For low-stakes, well-tested, repetitive decisions, yes. For anything consequential, no. The error-compounding problem means high-stakes chains need a checkpoint, regardless of how skilled the prompting is.
Key Takeaways
- More steps do not mean better reasoning; over-decomposition grows the failure surface faster than it helps, so use the fewest distinct steps the problem needs.
- The model does not durably remember constraints across a long chain; restate governing rules at each major step to prevent drift.
- Step-by-step reasoning looks like an audit trail but is generated text that may not reflect the real basis for a decision, so verify outcomes independently.
- A chain that works on its original examples is not automatically universal; test against deliberately varied inputs before trusting reuse.
- No amount of prompting skill removes compounding error, so mature practice pairs strong prompts with explicit checkpoints rather than running chains unsupervised.
For the practices that replace these myths, see The Hidden Risks of Prompting for Sequential Decision Making, Building a Repeatable Workflow for Prompting for Sequential Decision Making, and the full Prompting for Sequential Decision Making Playbook.