A 12-person AI agency in Portland built a document classification model for a legal services company in early 2024. The model ran in production for 18 months, processing roughly 4,000 documents per week. When the agency upgraded to a newer architecture that offered 23% better accuracy and significantly lower inference costs, they wanted to deprecate the old model. The problem: there was no deprecation policy. No defined timeline. No migration plan. No client communication protocol. The agency sent an email to the client saying the old model would be shut down in 30 days. The client pushed back hard — they had downstream systems, compliance reporting, and operational workflows that depended on the model's specific behavior and output format. The "30-day deprecation" turned into a seven-month migration project that cost the agency $95,000 in unplanned engineering time and severely strained the client relationship.
AI models are not permanent. They degrade. They get outperformed by newer approaches. Their underlying dependencies get deprecated. The data distributions they were trained on shift. Regulations change and render existing models non-compliant. Every model you build will eventually need to be retired.
Without a deprecation policy, every model sunset becomes an ad-hoc negotiation with the client. With a policy, deprecation is a planned, communicated, and managed process that maintains client trust while allowing your agency to evolve its technology stack.
Why AI Models Need Formal Deprecation Policies
Models Degrade Over Time
AI models are trained on data from a specific point in time. As the real world changes, the model's training data becomes less representative of current conditions. This is model drift, and it affects every production AI system.
- Customer behavior changes — a recommendation model trained on pre-pandemic shopping data underperforms on current purchasing patterns
- Language evolves — a natural language processing model trained on 2023 text mishandles terminology that emerged in 2025
- Business processes change — a classification model trained on historical workflows misclassifies documents when workflows are updated
- Data distributions shift — a fraud detection model trained on historical fraud patterns misses new fraud techniques
Model drift is not a failure — it is a natural consequence of deploying AI in a changing world. Deprecation policies ensure that degrading models are retired before they cause harm.
Dependencies Get Deprecated
Your AI models depend on third-party services that have their own deprecation cycles.
- Foundation model providers deprecate API versions and model generations
- Cloud services retire machine learning services and instance types
- Software libraries drop support for older versions
- Hardware architectures become unavailable
When a dependency is deprecated, every model that relies on it needs attention. A deprecation policy ensures you handle these cascading deprecations systematically rather than reactively.
Better Models Replace Old Ones
AI capabilities advance rapidly. A model that was state-of-the-art when you built it may be significantly outperformed by newer approaches within months. Maintaining old models alongside better alternatives wastes operational resources and may expose clients to inferior performance.
Compliance Requirements Change
Regulatory frameworks for AI are evolving. A model that was compliant when deployed may not meet new regulatory requirements for transparency, fairness, or documentation. Deprecation policies ensure that non-compliant models are retired on appropriate timelines.
The Model Deprecation Policy Framework
Deprecation Triggers
Define the conditions that trigger a model deprecation review. Not every trigger leads to immediate deprecation — but every trigger should initiate an assessment.
Performance triggers:
- Model accuracy drops below defined thresholds
- Model latency exceeds acceptable limits
- Model error rate on specific categories exceeds defined limits
- Monitoring detects significant model drift
- Client complaints about output quality increase
Dependency triggers:
- Underlying foundation model is deprecated by its provider
- Critical software dependency reaches end of life
- Cloud service used for hosting is being retired
- Hardware required for inference is being discontinued
Business triggers:
- A significantly better model is available (internal development or market availability)
- The use case the model serves is no longer relevant to the client
- The client's business requirements have changed substantially
- Contract renewal provides a natural transition point
Compliance triggers:
- New regulations require capabilities the model does not have (explainability, fairness metrics)
- Audit findings identify compliance gaps in the model
- Privacy regulations require changes to data handling that affect the model
- Industry standards change in ways that affect model requirements
Deprecation Timeline Standards
Define standard timelines for model deprecation based on the urgency of the trigger.
Emergency deprecation (0-7 days):
- Security vulnerability in the model or its dependencies
- Model producing harmful or dangerous outputs
- Regulatory order requiring immediate action
- Critical dependency failure with no workaround
Emergency deprecation requires immediate client notification and may involve temporary service interruption. Define fallback procedures (manual processing, alternative services) for emergency deprecation scenarios.
Accelerated deprecation (30-60 days):
- Significant performance degradation affecting client operations
- Dependency deprecation with a short timeline
- Compliance gap identified with a defined remediation deadline
- Client request for accelerated transition
Standard deprecation (90-180 days):
- Planned model upgrade or replacement
- Gradual performance degradation that is manageable but trending negatively
- Dependency deprecation with a comfortable timeline
- Business decision to consolidate or rationalize model portfolio
Extended deprecation (180-365 days):
- Legacy models with deep client integration
- Models serving regulated workloads with compliance transition requirements
- Large-scale models with significant downstream dependencies
- Contractually mandated minimum support periods
Client Communication Protocol
How you communicate deprecation to clients determines whether it is a managed transition or a relationship crisis.
Deprecation announcement:
- Notify the client in writing as soon as the deprecation decision is made
- Explain the reason for deprecation (be specific and honest)
- Provide the deprecation timeline with key milestones
- Describe the replacement model or alternative (if available)
- Outline the migration support your agency will provide
- Identify potential impacts on the client's operations
Ongoing communication:
- Provide regular status updates throughout the deprecation period
- Notify the client of any timeline changes (earlier or later)
- Document and communicate any behavioral differences between the old and new models
- Provide testing windows for the client to validate the replacement model
- Escalate communication to client leadership if migration is at risk
Post-deprecation communication:
- Confirm successful migration and model retirement
- Provide performance comparison data (old model vs. new model)
- Document any issues encountered during migration and their resolution
- Update all relevant documentation and contracts
Migration Support Standards
Define what migration support your agency provides during deprecation.
Standard migration support includes:
- Technical documentation for the replacement model (API specifications, output format changes, behavioral differences)
- Migration guide with step-by-step instructions for transitioning
- Testing environment where the client can evaluate the replacement model before cutover
- Parallel running period where both old and new models are available
- Engineering support for troubleshooting migration issues
- Post-migration monitoring to verify the replacement model performs as expected
Enhanced migration support (additional cost or as part of premium support):
- Dedicated migration engineer assigned to the client
- Custom integration work to accommodate client-specific dependencies
- Extended parallel running period
- Guaranteed response times for migration support requests
- On-site support for complex migrations
Data Handling During Deprecation
Model deprecation raises data handling questions that need clear policies.
Training data:
- Define what happens to training data when the model is deprecated
- If the training data was client-provided, ensure it is returned or destroyed per the data sharing agreement
- If the training data includes licensed data, ensure license compliance continues through deprecation
- Document training data disposition for audit purposes
Model artifacts:
- Define retention periods for deprecated model weights and configurations
- Consider retaining deprecated model artifacts for a defined period for rollback capability
- Implement secure disposal of model artifacts after the retention period
- Document model artifact disposition
Operational data:
- Define what happens to logs, monitoring data, and operational records from the deprecated model
- Ensure compliance with data retention requirements
- Archive data needed for audit or dispute resolution
- Delete data that is no longer needed
Implementing Deprecation Policies in Practice
The Model Registry
A model registry is the operational backbone of your deprecation policy. Every production model should be registered with:
- Model identifier and version
- Deployment date and environment
- Client and project association
- Current status (active, deprecated-pending, deprecated, retired)
- Deprecation trigger date (if applicable)
- Planned retirement date (if applicable)
- Dependency inventory (foundation models, libraries, infrastructure)
- Performance metrics and monitoring dashboard link
Deprecation Workflow
Step 1: Trigger identification — A deprecation trigger is identified through monitoring, dependency notification, business decision, or compliance review.
Step 2: Impact assessment — Assess the impact of deprecation on clients, operations, and dependencies. Identify affected systems, stakeholders, and timelines.
Step 3: Deprecation plan — Create a specific plan for this deprecation, including timeline, migration approach, client communication, resource requirements, and risk mitigation.
Step 4: Approval — Submit the deprecation plan for approval by the appropriate authority (technical lead, account manager, or executive, depending on impact).
Step 5: Client notification — Communicate the deprecation to affected clients per the communication protocol.
Step 6: Migration execution — Execute the migration plan, providing support as defined in the policy.
Step 7: Parallel running — Run old and new models in parallel for the defined period, monitoring for issues.
Step 8: Cutover — Transition from old model to new model. Implement rollback procedures in case of issues.
Step 9: Retirement — After successful cutover and stabilization period, retire the old model. Handle data disposition per policy.
Step 10: Documentation — Document the deprecation process, outcomes, lessons learned, and updated model registry status.
Contractual Integration
Your deprecation policy should be referenced in client contracts.
Key contractual provisions:
- Define minimum notice periods for model deprecation
- Specify migration support obligations
- Address cost allocation for migration (included in support fees vs. additional charges)
- Define SLA adjustments during migration periods
- Include force majeure provisions for emergency deprecation scenarios
- Reference the deprecation policy as an exhibit to the contract
Common Deprecation Mistakes
Mistake 1: No advance communication. Surprising clients with deprecation notices destroys trust. Communicate early, even before final decisions are made.
Mistake 2: Deprecating without a replacement. If possible, never deprecate a model without a qualified replacement ready. If no replacement is available, communicate honestly about alternatives and timelines.
Mistake 3: Underestimating client dependencies. Clients build workflows, reports, and integrations around your model's specific behavior. Map these dependencies before planning deprecation.
Mistake 4: Insufficient testing of the replacement. The replacement model needs thorough testing in the client's specific context, not just general benchmarks.
Mistake 5: Forgetting about data. Model deprecation triggers data handling obligations. Training data, operational data, and model artifacts all need documented disposition.
Your Next Step
Inventory every production model your agency operates. For each model, assess the current deprecation risk — is the model degrading, are dependencies approaching end of life, are better alternatives available? Identify the models most likely to need deprecation in the next 12 months.
Then draft your deprecation policy using the framework outlined above. Start with the five core sections: deprecation triggers, timeline standards, client communication protocol, migration support standards, and data handling procedures. Socialize the policy with your team and incorporate it into your next client contract renewal.
The Portland agency's seven-month, $95,000 unplanned migration could have been a three-month, $25,000 planned transition with a proper deprecation policy. Build the policy before you need it.