A refinement loop is supposed to converge. You generate, critique, revise, and the output gets steadily better until it is good. When loops fail, they do not usually fail loudly. They fail by circling: each iteration feels like progress, but the output never actually arrives, and you eventually settle for something out of fatigue rather than satisfaction.
The good news is that loop failures cluster into a small number of recognizable patterns. Once you can name them, you can catch yourself mid-loop and correct course. This article walks through seven of the most common, explaining why each one happens, what it costs, and the specific practice that fixes it.
These are drawn from watching people work, not from theory. If you want the positive version of these corrections rather than the failure framing, Prompting for Iterative Refinement Loops: Best Practices That Actually Work gives the same ideas as habits to build.
Mistake One: Looping Without a Standard
The most common and most damaging mistake is starting to refine without defining what good looks like.
Why it happens
It feels productive to dive in. Writing a standard first feels like overhead when you are eager to see output.
What it costs
Without a standard, every draft silently redefines the goal. You drift, polishing toward a target that keeps moving, and the loop never finishes because there is no finish line.
The fix
- Write three to five checkable qualities before generating.
- Hold every draft against that list, not against your shifting gut.
- Treat the standard as the finish line it is.
Mistake Two: Vague Critique
The second failure is critiquing with impressions instead of specifics.
Why it happens
"This feels off" is the honest first reaction, and it is easy to stop there.
What it costs
Vague critique produces vague revision. "Make it better" gives the model nothing to act on, so the next draft changes things you did not want changed and misses the real problem.
The fix
- Point to a location: which sentence, which section.
- Name the gap against a specific standard item.
- State the direction the fix should go.
This is the same precision the step-by-step procedure builds into its compare step.
Mistake Three: Changing Everything at Once
Batching many changes into one revision is the failure that hides all the others.
Why it happens
It feels efficient to fix the whole failure list in a single instruction.
What it costs
When the result improves, you do not know which change helped. When it gets worse, you cannot cleanly back out. Interactions between changes stay invisible, and you lose the ability to learn from each step.
The fix
- Fix one failure per iteration.
- Verify it before moving to the next.
- Accept that one-at-a-time feels slower but converges faster.
Mistake Four: Trusting the Model's Self-Critique Blindly
Asking the model to critique its own work is useful, until you treat its verdict as final.
Why it happens
The model's self-critique is fluent and confident, which reads as authoritative.
What it costs
Models can rationalize their own drafts and miss the issues a human would catch. Outsourcing the standard to the model lets the loop converge on something the model approves of but the reader does not.
The fix
- Use the model's self-critique as a candidate list, not a verdict.
- Keep the standard and the final judgment human.
- Cross-check its critique against your own pass/fail list.
Mistake Five: Re-Rolling Instead of Steering
When a draft disappoints, some people just regenerate and hope for better luck.
Why it happens
Re-rolling is effortless. It requires no critique, no thought about what is wrong.
What it costs
Re-rolling is random. You might land somewhere better or worse, but you are not steering, so you do not converge. You can re-roll twenty times and never get closer to your actual standard.
The fix
- When a draft disappoints, critique it instead of discarding it.
- Feed the specific gap back as a targeted instruction.
- Reserve re-rolling for when a draft is so far off it is not worth reacting to.
The difference between steering and rerolling is the core lesson of Refining Model Output by Looping: A Plain Introduction.
Mistake Six: Never Stopping
The polish trap catches people who enjoy the loop too much.
Why it happens
Each tiny tweak feels like an improvement, and there is always one more thing you could adjust.
What it costs
Once the output clears your standard, further loops trade real time for gains nobody will notice. You burn hours polishing past the point of value.
The fix
- Stop when every standard item passes.
- Stop when successive revisions produce no noticeable improvement.
- Let the standard be your finish line, which is another reason to define it first.
Mistake Seven: Losing the Thread Across Iterations
In a long loop, people forget what they already fixed and what they were aiming for.
Why it happens
The loop lives in working memory, and working memory leaks over many iterations.
What it costs
You re-introduce problems you already solved, re-litigate decisions, and lose track of which standard items still fail. The loop circles because you keep undoing your own progress.
The fix
- Keep the standard visible throughout.
- Track which items pass and which still fail.
- After each revision, update that pass/fail picture so you always know where you are.
How These Mistakes Compound
The seven mistakes are not independent; they reinforce each other, which is why a loop can go badly wrong fast.
The chain reaction
- No standard makes critique vague, because there is nothing specific to critique against.
- Vague critique invites batched changes, because if you cannot name one problem you tend to gesture at all of them.
- Batched changes hide regressions, so you lose the thread and re-introduce solved problems.
- With no standard, you cannot tell when to stop, so the loop never ends.
Why fixing the root helps everything
Because these mistakes chain off the missing standard, fixing that one root cause defuses most of the others at once. A concrete standard makes critique specific, which makes one-change-at-a-time natural, which keeps regressions visible, which preserves the thread, which gives you a finish line. The corrections are not seven separate disciplines; they are mostly downstream of one.
A Quick Self-Diagnosis
When a loop feels like it is going badly, a short diagnostic tells you which mistake you are making.
Ask yourself
- Can I name what good looks like in checkable terms? If not, you are missing a standard.
- Is my last critique specific to a location? If not, you are critiquing vaguely.
- Did my last revision change one thing or several? If several, you are batching.
- Am I closer than three iterations ago? If you cannot tell, you have lost the thread.
- Has the output already met my standard? If so, you are in the polish trap.
Running this diagnostic mid-loop takes thirty seconds and almost always surfaces the specific habit that needs correcting, rather than leaving you with a vague sense that the loop is just hard.
Frequently Asked Questions
Which of these mistakes is the most damaging?
Looping without a standard, because it enables most of the others. With no standard you cannot critique specifically, cannot know when to stop, and cannot track progress. Fixing this one mistake prevents several others from ever taking hold.
How do I catch myself making these mid-loop?
Watch for the symptoms: the loop feels long, your critiques are getting vaguer, and you are not sure if you are closer than three iterations ago. Those are signs of a missing standard or batched changes. Pause and rewrite the standard, then resume one change at a time.
Is using the model's self-critique always a mistake?
No, only trusting it blindly is. Self-critique is a useful source of candidate issues, especially for things you might miss. The mistake is treating its verdict as final. Keep the standard and the final call human, and the model's critique becomes a helpful input.
How is re-rolling different from generating a probe draft?
A probe draft is a deliberate starting point you intend to react to. Re-rolling is discarding a draft and hoping the next is better without saying what was wrong. The first is the start of steering; the second is gambling. Probe once, then steer.
Why does changing one thing at a time matter so much?
Because it preserves cause and effect. When you change one thing, an improvement or regression is attributable, so you learn from every step and can cleanly undo mistakes. Batching changes hides interactions and makes the loop impossible to reason about, which is why it underlies so many failures.
What is the simplest single habit that prevents most of these?
Write a concrete standard before you generate. It gives you a finish line, makes critique specific, and lets you track progress. That one habit defuses the worst mistake and makes the others far easier to avoid.
Key Takeaways
- Looping without a standard is the root failure that enables most of the others.
- Vague critique produces vague revision; point to a location, a gap, and a direction.
- Changing one thing at a time preserves cause and effect and is the rule that prevents circling.
- Treat the model's self-critique as a candidate list, not a verdict, and keep the standard human.
- Stop when the standard passes; the polish trap trades real time for invisible gains.