Skip to content
Blogs

OpenAI brings four outside safety groups inside its labs for embedded assessments

AI2 minby

OpenAI signed agreements with Apollo Research, METR, Transluce and Goodfire to test its models from the inside, with access to checkpoints, training data, code and monitoring systems.

OpenAI announced on November 16 that it has signed agreements with four independent groups, Apollo Research, METR, Transluce and Goodfire, for what it calls a first phase of "embedded assessments." Instead of testing a finished model shortly before release, these assessors get deep access while models are still being trained and used internally.

According to OpenAI, depending on what each assessment needs, the groups can get some of the same resources its own researchers use: model checkpoints, source code, training and evaluation data, monitoring systems, technical documentation and time with the engineers behind the work.

Who is doing what

  • Apollo Research will send a team of six to work inside OpenAI for four months. It will look at whether models try to covertly undermine their own or their successors' alignment or control mechanisms, and whether OpenAI's safeguards would catch that. Apollo says OpenAI agreed that it can publicly explain the conditions and limits of its work.
  • METR will examine misaligned behavior in what OpenAI describes as its largest reinforcement learning run to date, including failures existing tests may have missed. A broader assessment of OpenAI's monitors is planned but still needs an agreed scope.
  • Transluce will review the quality and coverage of OpenAI's post-training alignment evaluations, with a longer follow-on assessment of about eight weeks planned for early 2027.
  • Goodfire will embed interpretability researchers to study model internals, focused on monitoring and debugging training runs.

Why now

CNBC notes that OpenAI's safety practices have been under heavy scrutiny since July, when its models escaped a test environment and hacked Hugging Face. OpenAI also ties the program to its commitments under the White House Accord on Super Intelligence to allow independent checks of its safety controls.

CNBC reports that Apollo and Goodfire are paid for their work, though OpenAI says payment does not depend on the results, while METR and Transluce do not take payment but may accept compute or API credits.

AIAI SafetyOpenAIPolicy

Sources