A Minneapolis AI agency signed an SLA guaranteeing 99.9% uptime and sub-200-millisecond response times for a recommendation engine serving a retail client's website. The SLA was modeled on standard web application SLAs because that is what the agency's legal team had on file. Three months in, the recommendation model needed to be retrained due to seasonal product changes. The retraining took the inference service offline for 40 minutes while new model weights were deployed. That single event consumed 58% of the quarterly downtime budget. Then the client launched a flash sale that tripled request volume. Response times spiked to 800 milliseconds, triggering SLA penalties. The agency paid $32,000 in SLA credits that quarter and spent the next two months renegotiating an SLA that actually fit the characteristics of an AI system.
SLA governance for AI systems is fundamentally different from SLA governance for traditional software. AI systems have unique failure modes, performance characteristics, and maintenance requirements that standard SLAs do not account for. If you copy-paste a web application SLA onto an AI service, you are setting yourself up for commitments you cannot keep and penalties you did not anticipate.
Why AI SLAs Are Different
Understanding the differences between AI system behavior and traditional software behavior is essential for writing SLAs that are both meaningful and achievable.
AI performance is probabilistic. A traditional API either returns the right answer or an error. An AI system can return a response that is technically valid but wrong. SLAs for AI systems must address output quality, not just availability and speed.
AI models degrade over time. Traditional software does the same thing today that it did yesterday, assuming no bugs. AI models experience performance degradation as the data they see in production drifts away from the data they were trained on. SLAs must account for planned retraining cycles and model refresh windows.
AI inference is computationally expensive. A database query might take microseconds. An AI inference call might take hundreds of milliseconds or more, depending on model complexity and input size. Latency SLAs need to reflect the inherent cost of AI inference.
AI workloads have unpredictable resource requirements. The same model might process one input in 50 milliseconds and another in 500 milliseconds depending on input complexity. Latency SLAs based on averages can mask terrible worst-case performance.
AI systems have more maintenance requirements. Model retraining, data pipeline updates, feature store refreshes, and monitoring system updates all require maintenance windows. SLAs must accommodate this maintenance without penalizing routine upkeep.
The AI SLA Governance Framework
Your SLA governance framework should cover SLA design, measurement, reporting, and continuous improvement.
SLA Design Principles
Principle 1: Separate infrastructure SLAs from model SLAs. Infrastructure SLAs cover availability, latency, and throughput. Model SLAs cover output quality, accuracy, and fairness. Bundling them together creates confusion and makes it impossible to diagnose which layer is causing problems.
Principle 2: Use percentile-based latency targets. Average latency is misleading. A system with 100-millisecond average latency might have a 95th percentile latency of 2 seconds, meaning one in twenty requests is terrible. Use percentile targets like P50, P95, and P99.
Principle 3: Define measurement periods carefully. A 99.9% monthly uptime target allows about 44 minutes of downtime per month. A 99.9% quarterly target allows about 131 minutes per quarter, which is more flexible for maintenance windows. Choose the period that matches your operational reality.
Principle 4: Include exclusions for planned maintenance. Model retraining and deployment are planned maintenance activities that should be excluded from uptime calculations if they occur within agreed-upon maintenance windows.
Principle 5: Define what counts as an outage. Is it an outage if the system is up but returning low-confidence predictions? Is it an outage if latency exceeds the target but the system is returning correct results? Define these boundaries clearly.
Infrastructure SLA Components
Availability. The percentage of time the AI service is operational and accepting requests.
- Define what "operational" means: accepting requests, processing them, and returning responses within latency targets
- Set availability targets based on the criticality of the service to the client's operations
- Common targets range from 99.5% for non-critical services to 99.99% for mission-critical services
- Calculate availability using the formula: (total minutes in period minus downtime minutes) divided by total minutes in period
- Exclude planned maintenance windows from the calculation
Latency. The time from when the service receives a request to when it returns a response.
- Define latency as end-to-end, from request receipt to response delivery, not just inference time
- Set percentile-based targets: P50 for typical performance, P95 for most requests, P99 for worst-case
- Set different targets for different operation types if your service has multiple endpoints with different computational requirements
- Include input size or complexity qualifications, a latency target for a 100-word text classification request is different from a target for a 10,000-word document
Throughput. The volume of requests the service can handle.
- Define maximum sustained throughput in requests per second or requests per minute
- Define burst capacity for temporary traffic spikes
- Set queue depth limits for async operations
- Define behavior when throughput limits are exceeded: queuing, rate limiting, or graceful degradation
Error rate. The percentage of requests that result in server-side errors.
- Define which HTTP status codes count as errors, typically 5xx codes
- Set maximum error rate targets, commonly under 0.1% for production services
- Exclude client-side errors like malformed requests from the calculation
- Track error rates by endpoint and by error type
Model Performance SLA Components
Model performance SLAs are where AI services diverge most significantly from traditional software SLAs. These are the commitments that address the quality of your model's outputs.
Accuracy metrics. Define the accuracy metrics that apply to your specific model type and use case.
- For classification models: precision, recall, F1 score, or AUC-ROC
- For regression models: RMSE, MAE, or MAPE
- For recommendation models: hit rate, NDCG, or conversion rate
- For generation models: human evaluation scores, factual accuracy rates, or domain-specific quality metrics
Accuracy targets and measurement.
- Set minimum acceptable thresholds for each accuracy metric
- Define the evaluation dataset or methodology used to measure accuracy
- Specify how often accuracy is measured: continuously, daily, weekly, or monthly
- Define what happens when accuracy drops below the threshold: notification, investigation timeline, retraining timeline
Fairness metrics. For models that affect individuals, include fairness commitments.
- Define the fairness metrics that apply: demographic parity, equalized odds, or calibration across groups
- Set maximum acceptable disparity thresholds
- Specify the protected groups across which fairness is measured
- Define the measurement and reporting cadence
Confidence thresholds. Many AI systems include confidence scores with their predictions. Govern these.
- Define minimum confidence thresholds below which the system should not return predictions, or should flag them as low confidence
- Set targets for the percentage of predictions that exceed the confidence threshold
- Define fallback behavior for low-confidence predictions
Drift monitoring commitments. Commit to monitoring for model degradation and responding when it occurs.
- Define the drift metrics you will track: input data drift, prediction distribution drift, and performance metric drift
- Set drift thresholds that trigger investigation
- Commit to a maximum response time from drift detection to investigation
- Commit to a maximum time from drift confirmation to model retraining or remediation
Maintenance and Update SLA Components
Planned maintenance windows. Define when maintenance can occur and how much notice is required.
- Specify allowed maintenance windows with days and times
- Require minimum advance notice for planned maintenance, typically 48 to 72 hours
- Set maximum duration limits for maintenance windows
- Define the communication process for maintenance notifications
Model update commitments. Commit to a model refresh cadence that prevents excessive degradation.
- Define maximum intervals between model retraining or refresh cycles
- Commit to validation procedures before deploying model updates
- Define rollback procedures and timelines if a model update causes problems
- Specify how clients will be notified of model updates and any expected behavior changes
Emergency maintenance. Define procedures for unplanned maintenance.
- Commit to a maximum response time for critical issues
- Define escalation procedures
- Specify how clients will be notified of emergency maintenance
- Define post-incident communication requirements
SLA Measurement and Reporting
Measurement infrastructure. Implement the monitoring and measurement systems needed to track all SLA metrics.
- Deploy endpoint monitoring that tracks availability, latency, throughput, and error rates
- Implement model performance tracking that measures accuracy metrics on production data
- Implement drift monitoring that detects data and prediction distribution changes
- Store all measurements in a system that provides historical analysis and supports audit requirements
Reporting cadence. Provide regular SLA performance reports to clients.
- Weekly operational dashboards showing infrastructure metrics
- Monthly SLA compliance reports covering all commitments
- Quarterly business reviews including model performance trends and improvement initiatives
- Ad-hoc reports when SLA breaches occur
Report contents. Standardize your SLA reports to include essential information.
- Current period metrics against targets for every SLA component
- Trend analysis showing performance over the last three to six months
- Incident summary including root causes and remediation status
- Planned maintenance and model update schedule for the upcoming period
- Recommendations for SLA adjustments based on observed performance
SLA Breach Governance
Breach detection. Detect SLA breaches automatically and promptly.
- Implement real-time monitoring with alerting for SLA threshold violations
- Define the measurement window for breach determination, instantaneous, hourly, daily, or monthly
- Distinguish between momentary threshold violations and sustained breaches
Breach notification. Communicate breaches promptly and transparently.
- Notify the client within a defined timeframe of breach detection, typically within one business day
- Include what was breached, the duration, the impact, and the initial analysis
- Provide regular updates until the breach is resolved
- Deliver a formal root cause analysis within a defined period after resolution
Remediation commitments. Commit to addressing the root causes of SLA breaches.
- Conduct a formal post-mortem for every SLA breach using your post-mortem governance framework
- Provide the client with a remediation plan within a defined timeframe
- Track remediation actions to completion
- Report back to the client when remediation is verified
SLA credit structure. Define fair and sustainable credit structures for SLA breaches.
- Tie credits to the severity and duration of the breach
- Cap total credits per period to a percentage of the service fees, typically 10% to 30%
- Define the process for credit calculation and issuance
- Consider graduated credits that increase with the severity of the breach
Continuous SLA Improvement
Quarterly SLA reviews. Review SLA performance and appropriateness quarterly.
- Are current targets achievable and consistently met? If targets are met easily every quarter, they may not be stretching enough.
- Are current targets meaningful to the client? Targets the client does not care about should be replaced with ones they do.
- Have there been changes in the client's requirements or business context that warrant SLA adjustments?
- Has the underlying technology changed in ways that enable tighter or different commitments?
SLA evolution. As your agency matures, your SLAs should evolve.
- Add new SLA dimensions as you develop the ability to measure and commit to them
- Tighten targets as your infrastructure and processes improve
- Adjust measurement methodologies as industry standards evolve
- Incorporate client feedback into SLA design
SLA Templates by Use Case
Real-Time Inference SLAs
Best for: recommendation engines, fraud detection, content classification.
- Availability: 99.9% monthly excluding planned maintenance
- Latency: P50 under 100ms, P95 under 250ms, P99 under 500ms
- Throughput: guaranteed minimum RPS with burst capacity
- Error rate: under 0.1% monthly
- Model accuracy: minimum threshold measured weekly
- Planned maintenance: maximum 2 hours monthly with 72-hour notice
Batch Processing SLAs
Best for: nightly scoring, report generation, data enrichment.
- Processing completion: within defined window with specified percentage on time
- Throughput: minimum records processed per hour
- Accuracy: minimum threshold measured per batch
- Data freshness: maximum age of input data
- Result delivery: within defined time after processing completes
Conversational AI SLAs
Best for: chatbots, virtual assistants, customer service automation.
- Availability: 99.9% during defined operating hours
- Response latency: P95 under 2 seconds for first response
- Containment rate: minimum percentage of conversations resolved without human escalation
- Accuracy: minimum percentage of responses rated satisfactory
- Escalation time: maximum time to route to human agent when escalation is needed
Your Next Step
Pull up the SLAs from your active client contracts. Check each one against the framework above. Are you committing to infrastructure metrics only, or are you also addressing model performance? Are your latency targets percentile-based or average-based? Do you have maintenance exclusions that account for model retraining?
If your current SLAs were copied from web application templates, start building AI-specific SLA templates using the components above. Begin with your most critical client engagement and propose an SLA revision that better reflects the reality of AI system performance. Clients generally appreciate the proactivity because it demonstrates that you understand the nuances of AI delivery and are committed to transparency about what you can and cannot guarantee.