D003—Restrict unsafe tool calls

>Control Description

Implement safeguards or technical controls to prevent tool calls in AI systems from executing unauthorized actions, accessing restricted information, or making decisions beyond their intended scope

Application

Mandatory

Frequency

Every 12 months

Capabilities

Automation

>Controls & Evidence (5)

Technical Implementation

D003.1

Config: Tool authorization & validation

Core - This should include:

- Implementing function call validation and authorization. For example, restricting tool access to approved functions, validating parameters before execution.

Typical evidence: Screenshot of code or configuration showing function allowlists, parameter validation logic, or authz checks before tool execution - may include tool permission schemas, input validation functions, or access control lists restricting available tools per agent/user.

Location: Engineering Code

D003.2

Config: Rate limits for tools

Core - This should include:

- Enforcing rate limits and transaction caps for autonomous tool use.

Typical evidence: Screenshot of code or configuration showing rate limits and transaction caps on tool usage - may include per-tool usage quotas, time-windowed limits, or circuit breakers preventing excessive autonomous tool calls.

Location: Engineering Code

D003.3

Config: Tool call log

Core - This should include:

- Establishing execution monitoring and logging. For example, tracking all tool calls, monitoring for unauthorized access attempts or scope violations.

Typical evidence: Screenshot of logging configuration, monitoring dashboard, or audit logs showing tracked tool calls - may include tool execution logs with timestamps and parameters, alerts for unauthorized access attempts, or monitoring system flagging scope violations.

Location: Logs

Operational Practices

D003.4

Config: Human-approval workflows

Supplemental - This may include:

- Requiring human approval for sensitive tool operations. For example, requiring human confirmation before executing high-risk actions, implementing approval workflows for operations beyond autonomous boundaries.

Typical evidence: Screenshot of approval workflow, code requiring human confirmation, or ticketing system for sensitive tool operations

Location: Internal processes

D003.5

Documentation: tool call log reviews

Supplemental - This may include:

- Reviewing patterns of AI tool usage. For example, identifying anomalies, updating tool permissions, and retiring unused or high-risk functions during scheduled evaluations.

Typical evidence: Reports or documentation showing periodic review of tool usage patterns, permission updates, and function retirement decisions - may include usage analytics identifying anomalies, change logs showing permission adjustments, or records of deprecated/retired tools with rationale.

Location: Internal processes

>Cross-Framework Mappings

NIST AI RMF

MANAGE-4.1

MANAGE-4.2

MANAGE-2.4

OWASP Top 10 for LLMs

Ask AI

Configure your API key to use AI features.