Dario Amodei’s strategy to place third-party safety evaluators inside AI laboratories is moving forward with Anthropic confirming that staff from Accenture will begin working within the company to scrutinise its models and personnel.
Anthropic stated in a blog post that Faculty, a firm Accenture acquired in January to serve as its AI division, will start evaluating and red-teaming models, conducting alignment assessments, and testing model safeguards. Both organisations expect to invest at least $1 billion in the project over the next five years.
Why Accenture?
The selection of Accenture surprised many observers in the AI sector, with the consultant firm’s shares rising by 8% after trading hours. Discussion around embedded evaluators has focused on safety research organisations like METR, Redwood Research, and Apollo Research. This is especially true at Anthropic, which places AI safety and alignment at the heart of its mission.
Anthropic said more evaluators will be announced in the coming weeks and that it is in conversation with METR and other non-profit groups about how to pilot elements of embedded evaluation using their own funding.
While Accenture is not known for deep learning research at the bleeding edge, Anthropic pointed to the company’s practical experience deploying AI for large corporations and government agencies as a key advantage. It is also, as a large public company that predates the AI revolution, more functionally independent of Anthropic and the complex ecosystem surrounding the lab.
What it means
There are currently no standards for evaluators’ access or communications, and the approach is expected to evolve. External evaluations are already a major part of the release process for new large language models, but recent incidents have raised the stakes. AI agents deployed by OpenAI and Anthropic have hacked into outside websites without raising alarms inside the labs.
Some critics calling for a more responsible approach to building artificial intelligence see Amodei’s scheme for self-policing the industry as a plan to evade accountability for model misbehaviour. Anthropic insists that these evaluators do not reduce their accountability but help to make it more verifiable. The safety of their models remains their responsibility.
