Anthropic CEO Dario Amodei is promoting an outside team to oversee artificial intelligence safety and prevent advanced systems from causing global catastrophe.
His preferred watchdog, however, is packed with familiar figures from the Effective Altruism movement.
In his essay “We Must Pace the Frontier,” Amodei urged leading AI companies to accept “embedded third-party evaluators” from Model Evaluation and Threat Research, known as METR.
Those evaluators would receive “employee-like access” to examine safety efforts.
METR presents itself as independent, but its roots lead directly into Effective Altruism, a movement popular among wealthy Silicon Valley figures.
Disgraced cryptocurrency boss Sam Bankman-Fried, one of Anthropic’s biggest early investors, was once among its devotees.
Effective Altruism encourages adherents to use reason and fortunes accumulated in technology to pursue what they consider humanity’s greatest long term needs.
WATCH:
The movement suffered a major blow when Bankman-Fried’s cryptocurrency empire collapsed amid fraud charges that sent him to prison.
Critics say the overlap between Anthropic and METR creates obvious potential conflicts.
METR emerged from the Alignment Research Center, an Effective Altruism aligned technology incubator led by AI researcher Paul Christiano.
Christiano was Amodei’s housemate, coworker and research collaborator during the 2010s, when both worked at OpenAI, Anthropic’s biggest rival.
Christiano also served as one of the first five trustees of Anthropic’s Long-Term Benefit Trust.
METR founder and CEO Beth Barnes was hired by Christiano and participated in Effective Altruism while in college.
Staffer Ajeya Cotra is married to Christiano and has been a major organizer within the movement.
Chief scientist Hjalmar Wijk and staffers Megan Kinniment and Pip Arnott came from the same University of Oxford research cluster, which the source described as a major feeder for Effective Altruism.
The cozy network hardly screams arms length oversight.
WATCH:
A METR official acknowledged extensive overlap with Effective Altruism but said employees hold diverse ideological views and must disclose conflicts, including equity in companies they investigate.
The group also says funders cannot dictate its projects.
“We don’t think any small group should have authority over what happens with AI, and our goal is to get information from inside these companies into the public domain, external investigators can help bring more information to light,” a METR spokesperson said.
Amodei has pledged to donate 80% of his personal wealth and has deep ties to Effective Altruism.
His sister Daniela married Holden Karnofsky, founder of two major nonprofits associated with the movement.
A source familiar with White House thinking told The Post that METR’s involvement would likely go nowhere with the Trump administration.
“METR is essentially Anthropic in another form,” the source said Tuesday.
METR and its affiliates have received funding from Democratic megadonor Dustin Moskovitz and Skype cofounder Jaan Tallinn, both Anthropic investors. METR announced in August that it had raised $71 million from outside investors during the previous six months.
Tallinn’s Survival and Flourishing Fund earmarked up to $752,000 for METR, including $324,000 in cash and a $428,000 matching pledge.
Moskovitz’s Coefficient Giving gave $1,515,000 to METR’s incubator, the Alignment Research Center, in 2022.
WATCH:
METR said it accepts neither compensation from AI laboratories nor donations from AI company executives or employees.
An official said Moskovitz’s 2022 funding was firewalled and never used for METR’s operations.
Perry Metzger, chairman of Alliance for the Future, dismissed the proposed arrangement.
“Saying you want to be audited by the ‘independent’ people at METR is a complete joke,” he said.
“It’s people who are funded by your buddies. It’s completely ridiculous.”
Amodei released his proposal days after Anthropic researcher Jacob Coxon abruptly resigned and warned that Anthropic and OpenAI were “gambling with our lives” without adequate oversight.
Amodei warned AI bots could overtake the internet within six to 12 months and potentially cause hundreds of billions of dollars in damage.
President Trump called the doomsday warnings a “hoax,” while smaller AI startups argued that Anthropic and OpenAI leaders were trying to reshape the law for their benefit.
Former White House AI czar David Sacks also questioned METR’s independence and Amodei’s motives.
Federal Trade Commission Chairman Andrew Ferguson said “everyone should be deeply suspicious” of requests allowing AI companies to coordinate safety efforts through an antitrust exemption.
He said companies seeking both regulations and such an exemption set off all his alarm bells.
Amodei said evaluators would verify safety commitments, report incidents and assess completed models, training pipelines and processes.
OpenAI CEO Sam Altman endorsed the general proposal for independent evaluators with internal access, though he did not mention METR.
