Most teams approach fairness ad hoc: they remember to check for bias when something prompts them, run whatever test comes to mind, and move on. That catches the obvious cases and misses the structural ones, because there is no system ensuring every entry point gets examined. A framework fixes this by turning fairness from an act of memory into a sequence of stages you cannot skip without noticing.
This article introduces DEFEND, a six-stage model that maps to where bias actually enters and how a model actually lives in production. It is not a new technique; it is a structure for organizing techniques you may already know, so that nothing falls through the gaps between disciplines. Each stage names what to do, when it applies, and what failure it guards against.
D: Define the Fairness Goal
The framework starts before any building, with an explicit decision about what fair means here.
When and why
Apply this at project kickoff, in the spec. Because demographic parity, equalized odds, and predictive parity cannot all hold when base rates differ, you must choose one and document why. Defining the goal up front prevents the all-too-common failure of choosing whichever metric the finished model happens to pass. The main guide explains why these definitions conflict.
E: Examine the Data
The second stage scrutinizes the data and labels before modeling.
When and why
Apply this once you have a candidate dataset. Check representation across groups, interrogate how labels were produced, and hunt for proxy features. This is the highest-leverage stage because most bias enters here, and it is the one ad hoc approaches skip most often. Retain the protected attribute for auditing even as you exclude it from inputs.
A practical anchor for this stage is the dataset datasheet: a short document recording the data's origin, time span, label source, group representation, and known gaps. Producing it forces the examination to actually happen rather than being waved through. If you cannot answer who labeled the data and which groups are underrepresented, you have not examined it, you have only assumed it was fine.
F: Frame the Target Honestly
The third stage validates that the thing you are predicting is the thing you mean.
When and why
Apply this alongside data examination. A model that predicts a proxy for the real goal, like spending instead of need, encodes bias no downstream fix can remove. Framing the target honestly is a one-time check with outsized consequences, since a wrong target poisons everything after it. The examples article shows a triage system that failed exactly here.
E: Evaluate Per Group
The fourth stage is measurement, always disaggregated.
When and why
Apply this on every evaluation, from the first model onward. Compute your key metrics per group, build per-group confusion matrices, check calibration, and report the gap between best and worst. Aggregate-only evaluation is the failure this stage exists to prevent. The step-by-step guide details the exact computations.
N: Neutralize the Gap
The fifth stage is mitigation, applied only after measurement reveals a real gap.
When and why
Apply this when evaluation shows a disparity worth closing. Trace the gap to its cause, then choose the cheapest effective intervention: data-level fixes first, training constraints next, output thresholds last and cautiously. Re-measure after each change because closing one gap can open another, and document the accepted trade-off. The best practices article covers this ordering.
D: Defend Over Time
The final stage keeps fairness alive after launch.
When and why
Apply this continuously in production. Put per-group metrics in live monitoring, set a drift threshold that triggers re-auditing, define a retraining trigger, and name an accountable owner. This stage guards against the silent regression of a launch-day-fair model as the world shifts. Without it, the previous five stages produce a snapshot that ages badly.
How the Stages Fit Together
DEFEND is sequential but looping. The first three stages happen once per project at the start; Evaluate and Neutralize iterate during development; Defend runs forever in production and feeds back into re-examination when drift triggers a new cycle. The value is not any single stage, most of which teams already know, but the guarantee that none gets skipped. When fairness is a named pipeline, "we forgot to check the labels" becomes visibly an omitted stage rather than an invisible gap.
The naming matters more than it seems. A vague intention to "be fair" cannot be audited, delegated, or tracked, because there is nothing concrete to point at. A six-stage model can be put on a project board, assigned owners, and reviewed stage by stage. It converts a value into a process, and processes are the only things organizations reliably execute under deadline pressure. When the schedule tightens, the team without a framework drops fairness first, because it was never a visible deliverable. The team with one drops it visibly, as a logged decision someone has to approve.
A Worked Pass Through the Stages
To make DEFEND concrete, walk it through a hypothetical model that recommends which customers to offer a retention discount. At Define, the team decides fairness here means equal accuracy across customer segments, because the goal is to identify genuine churn risk, not to equalize discount rates. At Examine, they discover one segment is thinly represented in the training data and that "churn" was labeled inconsistently across regions. At Frame, they confirm that "likely to churn" is the right target and not a proxy for something like "low historical spend," which would penalize a whole segment unfairly.
At Evaluate, per-segment metrics reveal the thin segment is predicted poorly, with churn risk systematically underestimated. At Neutralize, they fix the representation problem at the data level by augmenting that segment rather than hacking the output, and re-measure to confirm the gap narrowed without opening a new one. At Defend, per-segment accuracy goes into production monitoring with a drift threshold, and a named owner is scheduled to re-review quarterly. Each stage produced a specific, recorded decision, and skipping any one of them would have left a recognizable hole.
Frequently Asked Questions
Do I have to follow all six stages for every model?
Scale the rigor to the stakes, but never skip Define and Evaluate, which are cheap and catch the most. A low-stakes recommender might run a light version of Examine and Defend, while a lending model needs every stage in full. The framework's job is to make any skipped stage a conscious, documented choice rather than an accident.
How is DEFEND different from just running a bias checklist?
A checklist verifies individual items; a framework organizes them into a lifecycle with a clear order and feedback loop. The checklist tells you what to check; DEFEND tells you when each check applies and how the stages feed each other, including the loop back from production monitoring into re-examination. They complement each other: use the framework for structure and the checklist for completeness.
Which stage do teams most often skip?
Examine and Defend. Teams jump from defining a goal straight to evaluating the model, skipping the data audit, and they treat launch as the finish line, skipping ongoing monitoring. Those two omissions account for a large share of real-world fairness failures, which is why the framework names them as mandatory stages rather than optional extras.
Can this framework work for a solo practitioner?
Yes. The stages are about sequence and completeness, not headcount. A single person can walk a model through all six stages, switching perspectives between framing, data, and deployment. The discipline of naming the stages is arguably more valuable solo, because there is no teammate to catch a skipped step for you.
Key Takeaways
- DEFEND organizes fairness into six stages: Define, Examine, Frame, Evaluate, Neutralize, Defend.
- The first three stages run once at project start; Evaluate and Neutralize iterate; Defend runs continuously.
- The framework's value is guaranteeing no entry point for bias gets skipped, turning omissions visible.
- Examine and Defend are the most commonly skipped and the source of many real failures.
- Scale rigor to stakes, but never skip defining the fairness goal or evaluating per group.