Safety

AI-generated text

OpenAI brings in four independent groups for embedded safety assessments

OpenAI has signed agreements with Apollo Research, METR, Transluce and Goodfire to assess model alignment and safety controls using access to internal systems.

OpenAI brings in four independent groups for embedded safety assessments

OpenAI has signed agreements with four independent organizations—Apollo Research, METR, Transluce and Goodfire—for the first phase of safety assessments that give outside experts deeper access to its internal systems. The company says public reports from external evaluations can help people understand how AI labs assess and manage risks as capabilities advance.

Depending on the task, assessors may gain access to some of the internal resources used by researchers, including model checkpoints, source code, training and evaluation data, monitoring systems and technical documentation. They may also speak with the researchers and engineers responsible for the work. The initial assessments focus on alignment and safeguards during training and internal use, complementing OpenAI’s safety cases.

Four assessment assignments

Apollo Research will examine whether models could covertly act against their developers’ intentions and whether OpenAI’s safeguards would detect such behavior. Across two stages, it will assess four claims: training did not incentivize subversion; rigorous evaluations and red teaming do not elicit it; models have not attempted it during internal use; and reasoning behind it would be detectable. For the first stage, Apollo will receive access to selected public and unreleased models, training checkpoints, relevant data and monitoring systems, among other resources.

Goodfire’s interpretability researchers will use the organization’s platform to analyze model internals. Their work will focus on monitoring training runs, debugging and alignment. OpenAI researchers will be the primary users of the findings, but Goodfire will also publish lessons and findings as the collaboration progresses.

OpenAI intends to conduct two assessments with METR. In the first, METR will look for misaligned behavior in OpenAI’s largest reinforcement learning run to date, including failures existing tests and monitors may have missed. Within the agreed access and safety arrangements, METR will independently set priorities and refine its searches, analyses and experiments to elicit behavior. This first phase is narrower than the broad survey envisioned in METR’s own proposal.

The second, preliminarily agreed METR assessment would focus on monitoring, although its scope still needs to be settled. It follows a three-week monitoring red teaming exercise in which METR had access to OpenAI’s code, monitors, relevant operational trajectories and research staff.

Transluce will initially examine the quality and coverage of post-training alignment evaluations. It will have access to selected public and unreleased models, evaluations and their results, model behavior data and safety documentation. OpenAI and Transluce also intend to conduct a follow-on assessment of approximately eight weeks in early 2027, with broader model access and a look at how evaluation results inform release decisions.

Access and oversight

OpenAI developed its procedures with input from the assessors, aiming to balance deep access with security, confidentiality and independence. Each engagement begins with an agreed scope, resources and safety arrangements, but assessors can refine their methods and propose further work as evidence warrants. If direct access would expose particularly sensitive intellectual property, such as model weights or architecture, OpenAI will consider alternatives, including pairing assessors with its own researchers for analyses.

Assessors who need sensitive confidential information will work at OpenAI offices using company devices and follow at least the security standards applicable to employees; some experiments may require stricter rules. OpenAI also expects early challenges involving technical access, space and evolving security procedures.

The Safety and Security Committee, an independent oversight committee of the OpenAI Foundation’s board, will be able to monitor the assessments. It will receive unredacted reports and can speak privately with assessors. Disagreements between OpenAI and an evaluator will first be escalated to the Safety Advisory Group and company executives, with the committee notified; unresolved matters will be referred to the committee.

Conversations with employees can inform assessments but cannot be published or attributed to staff. Through a separate process, assessors can ask OpenAI to verify, correct or clarify a statement. They may publish statements the company attests to, or disclose that it declined to do so.

Independence and public reporting

Assessors retain independence in publishing their evaluations, while OpenAI reserves the right to redact confidential information. The company may also request delays in responsibly disclosing findings that present safety or security risks, allowing time to address problems. Alongside public reports, oversight bodies will receive more detailed confidential reports.

Apollo Research and Goodfire receive financial compensation that does not depend on assessment results. METR and Transluce do not accept payment, but may receive API credits or computing resources for their assessments. OpenAI says it is still developing conflict-of-interest procedures with the organizations; it also asks participants to observe a brief cool-off period before undertaking similar work with another AI developer.

The company says the initiative supports its commitment to independent safety assessments under the White House Accord on Super Intelligence. OpenAI hopes to establish sustained embedded assessments over time and support external evaluations of further safety questions in the coming months. Future work could also examine compliance with safety frameworks and the company’s organizational culture and processes.