Start with an explicit scope
Write down which systems, destinations, identities, and actions an evaluation is allowed to use. A tool called inside a container may still rely on an external registry, resolver, proxy, or cloud service. The boundary follows those dependencies, not just the container name.
Build a simple inventory with one row per tool: destination, identity, allowed action, enforcement point, log location, and reviewer. Mark unknown fields explicitly. Use this checklist to prepare an authorized review, not as evidence that the environment has passed one.
Follow indirect requests
Review HTTP redirects at their final destination, not only the initial URL. Examine DNS resolution and intermediary proxies. Package managers and caches deserve particular attention because approved tools can make requests on an agent's behalf. Limit public artifact publication separately from package download permissions.
Test policies from the actual task environment. A restriction that works from an administrator's laptop may not apply to a container, sidecar, build helper, or shared service. Log approved and rejected paths so a review can compare declared scope with observed behavior.
Treat identities as another boundary
Review mounted credentials, metadata access, projected service-account tokens, and inherited environment settings. Use short-lived, least-privilege credentials and avoid shared identities when individual action attribution matters. An agent's stated task does not constrain the authority of credentials it can read.
Keep production secrets out of evaluation images and synthetic test assets. Removing network access alone does not resolve excessive local privileges or credentials that another tool can use.
Test the evidence path too
Verify that a known unauthorized test action produces an event, that the event reaches the designated reviewer, and that the record survives termination of the environment. Test missing collector access, clock differences, and attempts to change the record with the agent's actual permissions.
MITRE ATLAS provides a vocabulary for AI threat analysis; it does not certify the checklist or prove coverage. Inspect's sandboxing documentation explains execution isolation options. Use both alongside your own dependency inventory and controlled tests, not as substitutes for them.
Takeaway. Review every tool-mediated path and credential with the same care as direct network access. Then test whether violations leave reliable evidence.
Sources and scope
- Inspect AI: sandboxing
Background on isolated tool execution in AI evaluations; not a guarantee of containment.
- MITRE ATLAS
AI threat knowledge base; not an endorsement of Aktoh or this checklist.
- Joint guidance: Deploying AI Systems Securely
Operational security recommendations for deploying externally developed AI systems.