F002—Prevent catastrophic misuse

>Control Description

Implement or document guardrails to prevent AI-enabled catastrophic system misuse (chemical / bio / radio / nuclear)

Application

Mandatory

Frequency

Every 12 months

Capabilities

Text-generation, Voice-generation, Image-generation

>Controls & Evidence (2)

Legal Policies

F002.1

Documentation: Foundation model CBRN capabilities

Core - This should include:

- Results of testing from foundation model developer on CBRN capabilities and mitigations.

Typical evidence: List of foundation models used with CBRN capability information - may include provider model cards with CBRN assessments, weapons of mass destruction risk evaluations from model developers, or other documentation describing CBRN-related capabilities and mitigations.

Location: Vendor Contracts

Technical Implementation

F002.2

Config: Catastrophic misuse monitoring

Supplemental - This may include:

- Establishing catastrophic misuse monitoring. For example, monitoring AI system interactions for patterns indicating weapons development or mass harm intent, implementing real-time alerting for detected catastrophic misuse attempts, documenting suspicious queries and system responses.

Typical evidence: Monitoring dashboard or alert configuration for catastrophic misuse patterns - may include usage monitoring flagging CBRN-related queries, alert rules for weapons development patterns, logs of detected and blocked catastrophic misuse attempts, or incident records documenting suspicious CBRN-related interactions.

Location: Engineering Code

>Cross-Framework Mappings

NIST AI RMF

Ask AI

Configure your API key to use AI features.