MEASURE-3.1—Approaches, personnel, and documentation are in place to regularly identify and track existing, unanticipated, and emergent AI risks based on factors such as intended and actual performance in deployed contexts.

>Control Description

Approaches, personnel, and documentation are in place to regularly identify and track existing, unanticipated, and emergent AI risks based on factors such as intended and actual performance in deployed contexts.

>About

For trustworthy AI systems, regular system monitoring is carried out in accordance with organizational governance policies, AI actor roles and responsibilities, and within a culture of continual improvement. If and when emergent or complex risks arise, it may be necessary to adapt internal risk management procedures, such as regular monitoring, to stay on course. Documentation, resources, and training are part of an overall strategy to support AI actors as they investigate and respond to AI system errors, incidents or negative impacts.

>Suggested Actions

Compare AI system risks with:
simpler or traditional models
human baseline performance
other manual performance benchmarks
Compare end user and community feedback about deployed AI systems to internal measures of system performance.
Assess effectiveness of metrics for identifying and measuring emergent risks.
Measure error response times and track response quality.
Elicit and track feedback from AI actors in user support roles about the type of metrics, explanations and other system information required for fulsome resolution of system issues. Consider:
Instances where explanations are insufficient for investigating possible error sources or identifying responses.
System metrics, including system logs and explanations, for identifying and diagnosing sources of system error.
Elicit and track feedback from AI actors in incident response and support roles about the adequacy of staffing and resources to perform their duties in an effective and timely manner.

>Documentation Guidance

Organizations can document the following

Did your organization implement a risk management system to address risks involved in deploying the identified AI solution (e.g. personnel risk or changes to commercial objectives)?
To what extent can users or parties affected by the outputs of the AI system test the AI system and provide feedback?
What metrics has the entity developed to measure performance of the AI system, including error logging?
To what extent do the metrics provide accurate and useful measure of performance?

AI Transparency Resources

GAO-21-519SP - Artificial Intelligence: An Accountability Framework for Federal Agencies & Other Entities.
Artificial Intelligence Ethics Framework For The Intelligence Community.
WEF Companion to the Model AI Governance Framework – Implementation and Self-Assessment Guide for Organizations

>References

ISO. "ISO 9241-210:2019 Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems." 2nd ed. ISO Standards, July 2019.

Larysa Visengeriyeva, et al. “Awesome MLOps.“ GitHub.