Zero-shot versus few-shot is usually framed as a two-way choice, but the real decision has three options once you include fine-tuning. Each wins on different axes, and pretending one is universally best is how teams end up over-paying or under-delivering. This article lays out the competing approaches, the axes that actually drive the decision, and a concrete decision rule you can apply without an afternoon of deliberation.
The honest summary up front: zero-shot is the cheapest and most flexible, few-shot trades tokens for accuracy on tasks with implicit rules, and fine-tuning trades setup cost for the best per-call economics on stable, high-volume tasks. The rest is knowing which axis dominates for your case.
The Three Options
Zero-shot sends an instruction and the input, no examples. Cheapest per call, most flexible, transfers across models. Its ceiling is tasks the instruction can fully specify.
Few-shot prepends examples to teach implicit rules. Higher per-call cost in tokens and latency, but lifts accuracy on tasks with schemas, voice, or styles that words struggle to convey. No setup cost beyond curating examples.
Fine-tuning bakes the behavior into the model weights. High setup cost and a training data requirement, but the lowest per-call cost at scale and the most consistent behavior. Locks you to a model until you retrain.
The Axes That Matter
Four axes decide the choice. Weighting them for your situation is the whole job.
Cost per call
Zero-shot wins. Few-shot adds example tokens to every request — often 1,000 to 2,000 — which compounds at volume. Fine-tuning has near-zero marginal prompt cost but a fixed training cost to amortize.
Accuracy on implicit-rule tasks
Few-shot and fine-tuning beat zero-shot when the task carries conventions hard to state in words. For tasks a clear instruction fully covers, all three converge and zero-shot wins on cost.
Flexibility and iteration speed
Zero-shot and few-shot win decisively. You change a prompt in seconds. Fine-tuning requires a retrain cycle for every behavior change, which kills it for evolving tasks.
Consistency at scale
Fine-tuning wins. It produces the most uniform behavior because the rules are in the weights, not re-interpreted from examples on every call. Few-shot output can drift with example order and recency bias.
The Decision Rule
Apply these in order; the first match is your answer.
- Is the task fully specifiable in a clear instruction? If yes, use zero-shot. Do not add examples to a task the instruction already covers — that is the most common waste, detailed in our common mistakes guide.
- Does the task carry implicit rules (schema, voice, code style) but evolve or run at low-to-moderate volume? Use few-shot. You get the accuracy lift without a training commitment, and you can iterate freely.
- Is the task narrow, stable, high-volume, and is the example token cost now large? Consider fine-tuning. At that point the fixed training cost amortizes below the ongoing token cost of carrying examples, and you gain consistency.
The rule is a loop, not a one-time decision. Re-run it on every model upgrade, because newer models push more tasks into category 1, letting you delete examples — exactly what happened in our case study.
Worked Example: Applying the Rule to Three Tasks
Abstract rules are easier to trust when you see them resolve real cases. Consider three tasks a team might run side by side.
A support-ticket classifier into eight categories: fully specifiable in a clear instruction once you define each category boundary. Rule step 1 fires — zero-shot. The team that reaches for examples here is paying tokens to teach the model something the instruction already covers.
A brand-voice email generator: the voice cannot be captured in adjectives, the campaigns change weekly, and volume is moderate. Step 1 fails (voice is implicit), step 2 fires — few-shot with three on-brand samples. Fine-tuning would be overkill given how often the voice and campaigns evolve.
A product-title normalizer running on two million records a month with a fixed, unchanging convention: implicit rules, stable, very high volume, and a now-significant example token cost. Steps 1 and 2 are outgrown; step 3 fires — fine-tuning. The training cost amortizes in weeks against the token savings, and consistency improves because the convention lives in the weights.
The same rule produces three different answers because the axes weight differently for each task. That is the point: there is no universal winner, only a winner per axis profile.
The Hidden Cost: Maintenance Burden
There is a fifth axis people forget — maintenance. Zero-shot prompts are nearly free to maintain; there is no example set to curate, balance, or refresh. Few-shot prompts carry ongoing curation work: examples drift out of date as your input distribution shifts, and a stale example set silently degrades accuracy. Fine-tuned models carry retraining burden: every behavior change means a new training run and a new evaluation cycle.
Factor this into the decision. A few-shot prompt that wins narrowly on accuracy but demands constant example curation may lose to zero-shot once you price in the engineering time. This is why we push teams toward zero-shot whenever the instruction can carry the task — the cheapest prompt to run is usually also the cheapest to maintain.
Where Teams Go Wrong on the Trade-off
The two recurring errors are over-using few-shot and under-using fine-tuning. Teams add examples reflexively to zero-shot-solvable tasks, paying tokens for nothing. And teams keep growing few-shot prompts on stable high-volume tasks long past the point where a fine-tune would be cheaper and more consistent. Both errors come from never measuring the token cost of examples.
A third, subtler error is treating the decision as permanent. The right answer for a task today may be wrong after the next model upgrade or after volume grows tenfold. Build the decision rule into a recurring review, not a one-time architecture choice. For how to instrument the cost side, see the metrics guide, and for the structured decision process, A Framework for Zero Shot vs Few Shot Learning.
Frequently Asked Questions
Is few-shot ever a permanent solution, or just a stepping stone to fine-tuning?
It is a permanent solution for evolving or low-to-moderate-volume tasks where iteration speed matters more than per-call cost. It becomes a stepping stone only when a task stabilizes and volume grows enough that example token costs exceed a fine-tune.
How do I weigh the axes for my specific task?
Ask which axis would hurt most if it lost. If per-call cost dominates and the task is specifiable, zero-shot. If accuracy on implicit rules dominates and the task evolves, few-shot. If consistency at high volume dominates and the task is stable, fine-tuning.
Why re-run the decision rule on every model upgrade?
Because newer models solve more tasks zero-shot, moving them into category 1 of the rule. Re-running lets you delete examples you no longer need, cutting cost for equal accuracy.
What makes fine-tuning more consistent than few-shot?
The rules live in the model weights rather than being re-interpreted from examples on every call. Few-shot output can shift with example order and recency bias, while a fine-tuned model behaves uniformly across requests.
Can I combine approaches?
Yes — a common pattern is fine-tuning for the stable core behavior and a small few-shot or instruction tweak for evolving edge cases. Combine when one approach alone leaves a specific axis underserved.
How does maintenance burden factor into the choice?
It is the fifth axis. Zero-shot is nearly free to maintain, few-shot requires ongoing example curation, and fine-tuning requires retraining for every behavior change. A few-shot prompt that wins narrowly on accuracy can lose once you price in the engineering time to keep its examples current.
Key Takeaways
- Three real options: zero-shot (cheapest, flexible), few-shot (accuracy on implicit rules), fine-tuning (consistent, best at scale).
- Four axes decide it: cost per call, accuracy on implicit-rule tasks, flexibility, and consistency.
- Decision rule: specifiable task to zero-shot, implicit-rule and evolving to few-shot, stable high-volume to fine-tuning.
- Re-run the rule on every model upgrade — newer models push tasks toward zero-shot.
- The two big errors are over-using few-shot and under-using fine-tuning, both from not measuring example cost.