AI Risk Management Research
- 3 minsThe papers below explore improvements to AI risk management. Except for a few parts of Barrett et al., I expect that they will all be comprehensible to public servants with a non-technical background (but if you fit that description and you disagree, please let me know).
A Methodology for Quantitative AI Risk Modeling, Murray et al., Dec 2025, https://arxiv.org/abs/2512.08844.
Introduces a framework for tying quantitative estimates of AI risks to particular AI capabilities, i.e. “if the AI succeeds at X, what is the danger of Y?” Elicits estimates via a modified Delphi process, then uses statistical methods to tie these estimates together and give probabilities for various outcomes.
Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse, Barrett et al., Dec 2025, https://arxiv.org/abs/2512.08864v1.
A concrete application of the previous paper—it applies Murray et al.’s methodology to an estimation of cybersecurity risks from AI. Folks without technical training may struggle to follow all of the methodology section, but this background is not required to read through the authors’ results.
The Role of Risk Modeling in Advanced AI Risk Management, Touzet et al., Dec 2025, https://arxiv.org/abs/2512.08723.
Another companion paper to Murray et al. which situates the authors’ methodology in the context of existing risk management techniques. Existing techniques covered include various forms of scenario building and methods for dealing with data scarcity.
The paper also summarizes techniques used in AI risk modelling to-date, and it surveys techniques used in other safety-critical industries such as nuclear energy and aviation. It notes that adaptation of existing techniques to AI analysis is highly complicated by the non-deterministic nature of AI systems, and that without research breakthroughs in verifiable AI safety, AI risk management cannot meet the standards typically expected of a high-risk industry.
The science and practice of proportionality in AI risk evaluations, Mougan et al., Feb 2026, https://arxiv.org/abs/2603.10017.
A short paper directly geared towards the EU Artificial Intelligence Act. It provides a brief analysis of the criteria required for risk management to conform with the EU’s principle of proportionality, under which all EU measures must be “suitable, necessary, and balanced.” It then applies this to several potential evaluation methods for assessing AI models’ cyber-offensive capabilities.
Safety Frameworks and Standards, Ziosi et al., Oct 2025, [link].
Compares AI labs’ own safety measures, e.g. Anthropic’s Responsible Scaling Policy, with international risk management standards, e.g. ISO standards, and makes a number of risk-management-standard-inspired suggestions for improving the former.
Open Problems in Frontier AI Risk Management, Ziosi et al., April 2026, https://arxiv.org/abs/2604.25982.
A “pick your research question” paper that compiles a long list of unresolved issues in AI risk management, grouped by “planning,” “identification,” “analysis,” “evaluation,” and “mitigation.” It primarily proposes issues related to lack of technical consensus, misalignment with existing risk management frameworks, and shortcomings in risk management implementation.
Prioritization of Risks from Artificial Intelligence, Saeri et al., June 2026, https://arxiv.org/abs/2606.04490.
A Delphi survey of experts on the likelihood of various AI risks, the populations most endangered by them, and the groups responsible for managing them. The paper considers the likelihood of risks both in a “business as usual” scenario and assuming the adoption of “pragmatic mitigations.”
Please feel free to contact me if you have questions on any of the above.
Alex Chalk
AI Research | Ontario Graduate Scholar | Software Developer