A healthcare-focused AI agency deployed a clinical decision support system that analyzed patient vitals and lab results to flag potential deterioration. The model had been validated extensively in development. Six months after deployment, a nurse reported that the system had failed to flag a patient who subsequently required emergency ICU transfer. Investigation revealed that the hospital had changed its lab equipment supplier three months earlier. The new equipment produced slightly different value ranges for several biomarkers. The model had been trained on the old ranges and was misinterpreting the new values. Nobody had identified "upstream data source changes" as a risk. Nobody had implemented monitoring for input distribution shifts. A foreseeable risk had been missed because the agency had no systematic risk identification process.
AI risk management is the systematic process of identifying, assessing, mitigating, and monitoring risks associated with AI systems. It is not about eliminating all risk—that is impossible. It is about understanding your risks, making informed decisions about which risks to accept, and implementing controls that keep the remaining risks within acceptable bounds.
The AI Risk Taxonomy
Before you can manage AI risks, you need to understand what kinds of risks exist. AI systems face risks across multiple dimensions.
Technical Risks
Model performance risks. The model does not perform as expected. This includes accuracy degradation, bias amplification, overfitting to training data, and failure on edge cases or out-of-distribution inputs.
Data risks. Problems with the data that feeds the model. This includes data quality issues, data drift (changes in input distributions over time), data poisoning (deliberate manipulation of training data), and data leakage (training data contaminating test data).
Infrastructure risks. Problems with the systems that host and serve the model. This includes availability failures, scaling issues, latency problems, and security vulnerabilities.
Integration risks. Problems at the boundary between the AI system and other systems. This includes API failures, data format mismatches, version incompatibilities, and timing dependencies.
Operational Risks
Human oversight failures. The human oversight mechanisms are inadequate, overloaded, or bypassed. Humans approve model outputs without meaningful review. Alert fatigue causes real issues to be ignored.
Process failures. Established processes for development, testing, deployment, or monitoring are not followed. Shortcuts are taken. Documentation is skipped. Reviews are rubber-stamped.
Vendor and supply chain risks. Third-party components, models, or data sources introduce risks. A vendor changes their API, discontinues a service, or experiences a security breach.
Staffing risks. Key personnel leave, taking critical knowledge with them. The team lacks the expertise needed to manage specific AI risks.
Compliance and Legal Risks
Regulatory non-compliance. The AI system violates applicable laws or regulations, such as GDPR, CCPA, HIPAA, the EU AI Act, or industry-specific regulations.
Contractual non-compliance. The AI system fails to meet contractual requirements, SLAs, or representations made to clients.
Intellectual property risks. The AI system infringes on third-party IP, or the agency's IP is inadequately protected. Training data includes copyrighted material used without authorization.
Liability risks. The AI system causes harm that results in legal liability. The allocation of liability between the agency and the client is unclear.
Reputational Risks
Bias and fairness incidents. The AI system produces unfair outcomes that attract public attention, media coverage, or regulatory scrutiny.
Privacy incidents. The AI system exposes personal information or uses it in ways that violate expectations.
Failure incidents. The AI system fails publicly, damaging confidence in the agency's competence.
Transparency failures. The AI system's behavior is opaque, and stakeholders feel misled about what it does or how it works.
The Risk Management Process
Step 1: Risk Identification
Systematically identify risks for each AI project. Use multiple identification techniques to ensure comprehensive coverage.
Brainstorming workshops. Gather the project team and relevant stakeholders for a structured brainstorming session. Walk through each risk category and identify specific risks for this project.
Checklist review. Use a standardized risk checklist that covers common AI risks across all categories. Review each item and assess its relevance to the current project.
Historical analysis. Review incidents from past projects—both your own and industry-wide. Identify whether similar risks exist in the current project.
Threat modeling. For security and safety risks, conduct structured threat modeling. Identify threat actors, attack vectors, and potential impacts.
Stakeholder interviews. Interview stakeholders including the client, end users, and domain experts. They often identify risks that the technical team overlooks.
Pre-mortem exercise. Imagine the project has failed catastrophically. Work backward to identify what went wrong. This technique is particularly effective at surfacing risks that teams are reluctant to acknowledge.
Document every identified risk. At this stage, do not filter or dismiss risks—cast a wide net and refine later.
Step 2: Risk Assessment
Assess each identified risk on two dimensions: likelihood and impact.
Likelihood assessment. Estimate the probability that the risk will materialize. Use a standardized scale:
- Very Low: Less than 5 percent probability
- Low: 5 to 20 percent probability
- Medium: 20 to 50 percent probability
- High: 50 to 80 percent probability
- Very High: Greater than 80 percent probability
Impact assessment. Estimate the consequences if the risk materializes. Assess impact across multiple dimensions:
- Financial impact (revenue loss, penalties, remediation costs)
- Reputational impact (client trust, market perception, media attention)
- Operational impact (service disruption, resource diversion, project delay)
- Legal impact (regulatory action, litigation, contractual claims)
- Human impact (harm to individuals, safety consequences, fairness violations)
Use a standardized scale:
- Negligible: Minimal consequences, easily managed
- Minor: Limited consequences, manageable with standard procedures
- Moderate: Significant consequences requiring dedicated response
- Major: Serious consequences with lasting effects
- Severe: Catastrophic consequences threatening business viability or causing serious harm
Risk rating. Combine likelihood and impact into an overall risk rating. A simple risk matrix works well for most agencies—map likelihood and impact to a grid that categorizes each risk as Low, Medium, High, or Critical.
Step 3: Risk Treatment
For each assessed risk, choose a treatment approach.
Mitigate. Implement controls that reduce the likelihood or impact of the risk. This is the most common treatment for AI risks. Mitigation may include technical controls (bias testing, monitoring, access controls), process controls (review procedures, approval gates, documentation requirements), and organizational controls (training, role assignments, escalation procedures).
Transfer. Shift the risk to another party through insurance, contractual allocation, or outsourcing. Professional liability insurance can transfer some financial risk. Contractual provisions can allocate risk between your agency and the client.
Avoid. Eliminate the risk by changing the approach, scope, or design. If a particular data source introduces unacceptable bias risk, avoiding that data source eliminates the risk. If a use case is inherently too risky, declining the engagement avoids the risk entirely.
Accept. Acknowledge the risk and choose to proceed without additional mitigation. This is appropriate for low-level risks where the cost of mitigation exceeds the expected impact. Acceptance must be a conscious, documented decision, not a default.
For each risk treatment, define the specific actions, the responsible parties, the timeline, and the success criteria.
Step 4: Risk Monitoring
After implementing risk treatments, monitor for changes in risk levels, new emerging risks, and the effectiveness of your controls.
Continuous monitoring. Implement automated monitoring for technical risks. Track model performance metrics, data quality indicators, infrastructure health, and security events. Configure alerts that trigger investigation when metrics breach defined thresholds.
Periodic reviews. Conduct regular risk reviews (monthly for high-risk projects, quarterly for others) that reassess risk levels, evaluate control effectiveness, and identify new risks.
Trigger-based reviews. Conduct additional risk reviews when significant changes occur, such as model updates, data source changes, scope changes, regulatory developments, or incidents.
Risk register updates. Maintain a living risk register that reflects current risk assessments, treatment status, and monitoring results. The risk register should be the single source of truth for AI risk information.
Step 5: Risk Communication
Communicate risk information to all relevant stakeholders in a format and frequency appropriate to their needs.
To the project team: Regular updates on risk status, emerging risks, and required actions. Include risk discussions in sprint planning and retrospectives.
To the client: Periodic risk reports that cover current risk levels, mitigation activities, and any concerns. Be transparent about risks—clients appreciate honesty more than false reassurance.
To leadership: Quarterly risk summaries that cover aggregate risk across the portfolio, trends, significant incidents, and resource needs.
To regulators: When required, provide risk information in the format and detail that regulators expect. Proactive engagement with regulators builds trust.
Building Your Risk Management Infrastructure
The Risk Register
The risk register is the central artifact of your risk management program. It should capture:
- Risk ID and description
- Risk category (technical, operational, compliance, reputational)
- Associated project or system
- Risk owner (the person responsible for managing the risk)
- Likelihood rating
- Impact rating
- Overall risk rating
- Treatment approach
- Specific mitigation actions and their status
- Residual risk after mitigation
- Monitoring approach
- Last review date
- Notes and history
Maintain the risk register in a tool that supports collaboration, filtering, and reporting. A spreadsheet works for small agencies. Larger agencies should consider dedicated GRC (governance, risk, and compliance) platforms.
Risk Assessment Templates
Create standardized templates for risk assessment activities including project risk assessment templates for new project kickoffs, change risk assessment templates for evaluating the risk of proposed changes, incident risk assessment templates for evaluating the risk implications of incidents, and periodic review templates for regular risk reassessments.
Risk Appetite Statement
Define your agency's risk appetite—the level of risk you are willing to accept in pursuit of your objectives. The risk appetite should be:
- Approved by leadership
- Specific enough to guide decisions (for example, "we will not accept high or critical risks related to patient safety" is more useful than "we have a low risk appetite")
- Differentiated by risk category (you may accept more technical risk than compliance risk)
- Reviewed and updated annually
Escalation Procedures
Define clear escalation procedures for risks that exceed your risk appetite. Specify who can approve risk acceptance at each level, how quickly escalation must happen, and what documentation is required.
Risk Management for Different Engagement Types
Custom Model Development
For custom model development projects, risk management focuses on model-specific risks: training data quality, model performance, bias, and deployment safety. Conduct a comprehensive risk assessment at project kickoff. Update the assessment as development progresses and design decisions are made.
Model Integration and Deployment
For projects where you integrate or deploy existing models (including third-party or open-source models), risk management focuses on integration risks, vendor risks, and deployment environment risks. Assess the model's fitness for the specific use case, even if it has been validated in other contexts.
Managed AI Services
For ongoing managed services (operating AI systems for clients), risk management focuses on operational risks: monitoring, incident response, change management, and performance maintenance. Implement continuous monitoring and regular risk reassessment.
AI Strategy and Advisory
For advisory engagements, risk management focuses on the quality of your advice and the risks of implementing your recommendations. Document the basis for your recommendations. Include risk assessments in your deliverables. Be explicit about assumptions and uncertainties.
Building a Risk-Aware Culture
From Compliance to Culture
Risk management that exists only as a compliance activity will be superficial and fragile. Build a culture where risk awareness is a natural part of how your team thinks and works.
Talk about risk openly. Include risk discussion in project meetings, sprint planning, and retrospectives. Make it normal to talk about what could go wrong.
Reward risk identification. Recognize team members who identify risks early. A risk identified before deployment is far less costly than a risk that materializes in production.
Learn from incidents. Conduct blameless post-mortems for all significant incidents. Share lessons learned across the team. Use incidents as opportunities to improve risk management practices.
Lead by example. Leadership must model risk awareness. When leaders take shortcuts on risk management, the team receives a clear signal that risk management is optional.
Integrating Risk Management Into Your Workflow
Risk management should be embedded in your existing workflow, not layered on top of it.
During project scoping: Conduct the initial risk assessment. The risk profile influences project planning, resource allocation, and pricing.
During sprint planning: Review the risk register. Assign risk mitigation tasks. Identify any new risks from the planned work.
During development: Follow risk-aware development practices. Implement monitoring and controls as you build. Document risk-relevant decisions.
During code review: Include risk considerations in code reviews. Does this change introduce new risks? Does it affect existing mitigations?
During deployment: Review the risk register before deployment. Ensure all identified risks have been addressed or accepted. Verify that monitoring and controls are in place.
During retrospectives: Review risk management effectiveness. Were risks identified early enough? Were mitigations effective? What can be improved?
Your Next Step
This week: Create a risk taxonomy relevant to your agency by adapting the categories in this playbook to your specific context. Conduct a risk identification exercise for your highest-risk active project. Document the identified risks in a risk register.
This month: Complete risk assessments for all active projects. Develop your risk treatment plans for high and critical risks. Implement the most urgent mitigations. Establish your risk appetite statement with leadership approval.
This quarter: Build risk management into your standard project workflow. Create assessment templates and checklists. Implement automated monitoring for key technical risks. Establish regular risk review cadences. Train your team on risk management practices and their specific responsibilities.