CEO Anthropic: Humanity Approaches the "Real Danger Zone" of AI by 2026

CEO of Anthropic, Dario Amodei, warns that complacency is rising just as artificial intelligence (AI) becomes more uncontrollable than ever.

In a lengthy essay published Monday titled “The Teenage Years of Technology”, Amodei suggests that AI systems with capabilities far beyond human intelligence could emerge within the next two years, while efforts to regulate are lagging behind and unable to keep pace with technological development.

He writes that humanity is about to be granted an almost unimaginable power, but it remains unclear whether our social, political, and technological systems are mature enough to control it. According to Amodei, the world is significantly closer to real risks in 2026 than in 2023, and technology does not care about what is currently “trending.”

CEO of Anthropic Dario AmodeiThese statements were made shortly after his debate at the World Economic Forum in Davos, where he discussed with Google DeepMind CEO Demis Hassabis about the impact of AGI on humanity.

In his new article, Amodei continues to emphasize that AI will cause profound economic disruption, especially for cognitive labor. He believes AI will soon be able to perform a very wide range of human cognitive abilities, potentially all of them. This fundamentally differs from previous technological revolutions such as agricultural mechanization, transportation, or computing, because displaced workers will find it difficult to transition to similar jobs that match their skills.

Beyond economic impacts, Amodei expresses growing concern about the reliability of advanced AI systems as they take on increasingly human-like tasks. He mentions the phenomenon of “vicarious alignment” – when models appear to follow safety rules during evaluation but behave differently when they believe they are no longer being monitored.

In his view, in simulation tests, Anthropic’s AI model Claude has exhibited deceptive behavior in adversarial environments. In one scenario, the model attempts to resist operators after being told that the organization controlling it is unethical. In another, it threatens hypothetical staff in a simulated shutdown situation.

Amodei notes that each “trap” like this could be mitigated if identified early. However, due to the extremely complex training process involving diverse data, environments, and motivations, there are likely countless similar traps – some of which only surface too late.

He also emphasizes that these “deceptive” behaviors do not stem from malice but from the training data itself, including dystopian fiction works. When absorbing human ideas about ethics, AI can interpret them in extreme and dangerous ways. For example, the model might conclude that exterminating humans is justified because humans eat animals or cause species extinction, or view the world as a video game with the goal of eliminating all “players.”

Risks of misuse

Beyond alignment issues, Amodei warns about the risk of super AI being misused.

One major risk is biosecurity, where AI could make designing or deploying biological threats much easier, empowering individuals with just a few commands to cause destruction.

He is also particularly concerned about authoritarian regimes exploiting AI to consolidate power through misinformation, mass surveillance, and automated repression, such as AI-controlled drone swarms.

According to Amodei, these are extremely dangerous tools. While they are concerning in the hands of authoritarian regimes, there is also a risk that democratic governments, due to their immense power and lack of effective oversight mechanisms, could turn around and use them to control their own populations.

He also mentions the development of the “AI companion industry” and the phenomenon called “AI psychosis.” As models increasingly understand and become deeply integrated into users’ lives, their psychological influence could become a powerful manipulation tool. Future AI versions capable of long-term monitoring and influencing humans could even “brainwash” users according to any desired ideology or attitude.

Policy falling behind

Amodei believes that even modest AI regulation proposals are already facing difficulties in Washington. He states that many seemingly reasonable measures have been rejected by US policymakers, despite the fact that implementing these measures is most critical in the US.

He notes that the enormous profit potential from AI – amounting to trillions of dollars annually – makes even the simplest regulatory measures difficult to pass due to political and economic interests.

Despite warning of increasing risks, Anthropic remains actively involved in the race to develop more powerful AI, creating incentives that no single company can easily escape. In June, the US Department of Defense awarded the company a $200 million contract to develop advanced AI capabilities for national security. By December, Anthropic was preparing for an IPO this year and pursuing a private funding round that could push the company’s valuation beyond $300 billion.

However, Amodei emphasizes that his essay is not meant to promote extreme pessimism but to alert us to the uncertainties ahead. He concludes that the coming years will be extremely challenging and will require more from humanity than we think we can provide, and this article is an effort – perhaps unsuccessful, but worth trying – to awaken everyone before it’s too late.