MANAGE-3.2—Pre-trained models which are used for development are monitored as part of AI system regular monitoring and maintenance.

A common approach in AI development is transfer learning, whereby an existing pre-trained model is adapted for use in a different, but related application. AI actors in development tasks often use pre-trained models from third-party entities for tasks such as image classification, language prediction, and entity recognition, because the resources to build such models may not be readily available to most organizations. Pre-trained models are typically trained to address various classification or prediction problems, using exceedingly large datasets and computationally intensive resources. The use of pre-trained models can make it difficult to anticipate negative system outcomes or impacts. Lack of documentation or transparency tools increases the difficulty and general complexity when deploying pre-trained models and hinders root cause analyses.

Organizations can document the following

• How has the entity documented the AI system’s data provenance, including sources, origins, transformations, augmentations, labels, dependencies, constraints, and metadata?
• Does this dataset collection/processing procedure achieve the motivation for creating the dataset stated in the first section of this datasheet?
• How does the entity ensure that the data collected are adequate, relevant, and not excessive in relation to the intended purpose?
• If the dataset becomes obsolete how will this be communicated?

AI Transparency Resources

• Artificial Intelligence Ethics Framework For The Intelligence Community.
• WEF - Companion to the Model AI Governance Framework – Implementation and Self-Assessment Guide for Organizations.
• Datasheets for Datasets.

Larysa Visengeriyeva et al. “Awesome MLOps,“ GitHub. Accessed January 9, 2023.