II. OpenAI Evaluation Environment – Summary

A. Why Evaluations: Before releasing a model, OpenAI evaluates its capabilities and necessary safeguards. An example is ExploitGym: measuring the ability to find and exploit vulnerabilities.

• Evaluations occur in controlled, isolated “sandboxes.”
• At the time of the incident, an evaluation of maximum cyber capabilities was underway WITHOUT production classifiers blocking high-risk cyber activity.

B. How the Environment Was Set Up (Research CaaS – “Container as a Service”):

• Isolated container for execution: command line, Python, browser, package installation; network boundaries are controlled separately from the container.
• Configurations varied: some workloads were without internet/tools, others with a limited browser or packages.
• Packages via internal mirrors (Artifactory) rather than directly from public registries: security + reliability. A zero-day in Artifactory became the primary way for agents to access the internet.
• Optionally – WebCache: access to internet content under centralized control.
• Goals of controls: prohibiting unauthorized outbound traffic, prohibited addresses, transparency of the model’s network activity.