Anthropic Gives Outside Evaluators Employee-Like Access to Test Claude Risks

Anthropic says a team from independent AI evaluator METR has begun receiving employee-like access to examine risks around its most capable systems. That is an unusually concrete answer to a question many AI companies face: can outsiders test a lab’s safety claims with more than selected demonstrations and published summaries?

The access is not unlimited. Anthropic says customer information, privileged legal material and model architecture details remain outside it. But within those boundaries, employees are encouraged to share relevant information with METR as they would with a colleague working on alignment risk. METR is expected to publish the results of its assessments, according to Anthropic’s October 15 announcement.

That design matters because a model can look reassuring in a controlled public benchmark and behave differently when it is connected to tools, allowed to carry out multi-step tasks or exposed to situations its developers did not anticipate. An outside evaluator needs room to inspect incidents, challenge assumptions and examine the systems around a model, not merely send prompts to a public chatbot.

Anthropic says the arrangement follows its commitment to stronger third-party assessment. It is also working with SecureBio and Accenture and is discussing potential evaluations with other organisations. What the company has described is an access model, however, not an independent certification that its systems are safe. The scope of METR’s eventual findings, and what Anthropic changes in response, will be more revealing than the initial announcement.

There is a broader governance test here. Outside reviewers can only be credible if they can report uncomfortable findings without the company controlling the conclusion. At the same time, any frontier AI lab has legitimate reasons to protect customer data and sensitive security details. The hard work is defining exceptions narrowly enough that they do not swallow the review itself.

TechBooky recently examined European questions about AI agents reaching real systems. In that context, meaningful access for external evaluators is more than a public-relations gesture. It could become part of the evidence regulators and customers expect before powerful models are placed in consequential workflows.

For now, Anthropic has opened a door that other AI labs may be pressed to open as well. The public should judge the experiment by what METR is able to investigate and publish, not by the reassuring sound of the words ‘independent evaluation’ alone.