Anthropic delenda est
Anthropic is not "the good one".
Written with input from Alvaro Cuba.
People ask me: “Why are you attacking Anthropic if there are more evil AI companies?”
Anthropic is, I think, if not the most evil AI company, the most insidious AI company. Anthropic does the same harm as all the AI companies— making superintelligence— and its own unique harm.
Safety washing frontier AI development
First of all, what Anthropic is doing is evil. ANYONE trying to build superintelligence without scientific consensus on safety and the consent and representation of the public is evil.
Anthropic has popularized the premature idea that superintelligence can be built safely (as long as it’s by them). We don’t know that. They don’t know that. But by claiming they can make safe superintelligence with their Good ValuesTM, they introduce the greedy vision that their supporters can have the power of superintelligence without the inherent risk.
Anthropic’s safetywashing is extremely harmful. The existence of a self-proclaimed “good guy” in the bad bunch of AI developers helps the bad guys out a lot. Many people think something like “if the AI industry are not categorically bad, I can’t oppose them”, no matter how irresponsible and risky their behavior. This adds friction to the already uphill process of establishing who’s boss (our democracy) and getting regulation on the AI industry.
We have to draw clear bright lines. The good guys are the people pushing for caution, regulation, and a Pause. The bad guys don’t like that because they are developing superintelligence as fast as they can so they can to “win” the race to make dangerous AI that puts everyone at risk. The prize for “winning” is seemingly world domination, which would trample the rights of everyone else. Anthropic is one of the bad guys, maybe even the leading bad guy depending on the day. As much as they declare themselves “the good guys” (paging George Orwell) because they pay lip service to safety, they are arguably the worst guy.
Dario Amodei’s insidious spin
Dario Amodei planned to kick off a cold war between the US and China in which HE would be the winner. He has been introducing the topic that way to OpenAI and then Anthropic recruits since before AI was on the world stage.
Anthropic was not the first to talk of “pivotal acts”. That’s MIRI’s term for using superintelligence to keep the rest of the world from making superintelligence. MIRI used to have soft world domination schemes to build superintelligence itself and then commit a pivotal act— it’s something that was much discussed before my time in the AI Safety community of the 2010s. But whereas MIRI has updated to working toward a global treaty, Anthropic is living out what, from what I can tell, was Dario and Holden Karnofsky’s pivotal act plan from the beginning: become the darling of the US National Security apparatus to get favored developer status and then own the Earth once Claude is superintelligent. Anthropic’s fight with the DoW over control of Claude in the military is foreshadowing of a scaled-up Anthropic eventually being able to refuse to follow our elected government.
The narrative Dario developed to build Anthropic gave license to the entire scaling AI industry. His manipulative frames— e.g. the empirical paradigm, in which building AI is the only way to do AI Safety, and which gave permission to idealistic young EAs (who wanted more than anything to do 80,000 Hours’ top career, technical AI Safety) to do straight-up capabilities work for Anthropic— and use of selective counterfactuals— basically asserting that, because Anthropic are “the good guys”, them building AI crowds out bad people building it, completely ignoring what Anthropic could do to stop ASI being built at all— are weapons that the entire AI race field has eagerly adopted to excuse their own dangerous AI building.
Anti-regulation lobbying
Anthropic lobbies against AI regulation. There are a few cases where people want to give them credit for weakly indicating support for legislation, like SB 1047, but generally in these cases they didn’t actually offer formal support and behind closed doors it appeared to be a completely different story from their lobbyists. Dario kind of opposed the idea of AI regulation preemption when he could get credit in a live interview but didn’t clearly back even that weak stance up with real action. (Dario accused Sam Altman of only backing up Anthropic with the DoW to satisfy the OpenAI employees, but I think Dario is the one who has to worry about optics to his employees and this is why he occasionally does something ostensibly pro-regulation.)
Most often Anthropic and its employees just refuse to fight for regulation or safety in the political realm because they say they can’t. I was told they couldn’t agree to a conditional pause because it was “anti-competitive” 🤷♀️ They have dropped their own voluntary safety commitments (“RSPs”), which were supposedly the reason we should trust them, TWICE because they wanted to violate them.
Pretty much Anthropic makes sure they get maximum brownie points for even weakly pro-regulation takes while the majority of their behind-closed-doors lobbying is the same as the other AI companies— whatever’s good for business, where business=racing to build superintelligence first. They will frequently defend doing bad things they said they would never do because now they realize they need to do them to win the race. Even the things that supposedly make OpenAI and Dario’s nemesis Sam Altman in particular so bad, like making deals for Middle Eastern dictator money, Dario thinks are okay for Anthropic to do when they need money or resources or to cut corners, because Anthropic does it as “the good guys”.
#YesALLAICompanies
Anthropic must ALSO be felled. We pull punches if we avoid hitting them. If I only went for xAI, OpenAI, and DeepMind, like many twitterati suggest, then what if those did fall? Anthropic would be the last one standing, and we would have spent our effort winning their race to rule the world for them.
Only Pause can slay Anthropic. Other people are attacking X and Meta for chatbot impropriety or OpenAI for being audacious and having a lying CEO. Opposing the attempt to build superintelligence itself is how we fell Anthropic along with all the rest.
There are no “good guys” building superintelligence. Especially not Anthropic. Their rhetoric protects and intensifies the entire AI race. In convincing safety advocates to spare them, Anthropic sabotages the entire AI Safety movement and hijacks it for its own reckless efforts to build superintelligence.
We need to #PauseAI. ALL frontier AI. Anthropic delenda est.