AI leaders themselves call for a halt amid fears of losing control: cyberattacks, employment, and an economy at risk

Artificial intelligence has entered a new phase: its own proponents are starting to ask for the brakes to be applied. Dario Amodei, CEO of Anthropic, has called for a slowdown in the development of the most advanced models due to the risk that their capabilities may grow faster than the mechanisms designed to control them. A warning that has been echoed by Sam Altman, head of OpenAI, and Elon Musk, founder of SpaceX.

The fear is no longer limited to futuristic scenarios. Among the threats that concern tech leaders are the loss of control over systems, cyberattacks, bioterrorism, and the economic disruptions that increasingly powerful automation can cause. Amodei argues that AI can bring enormous benefits—from accelerating economic growth to helping combat diseases—but warns that a technological race based solely on moving faster and faster can end up generating risks that are difficult to contain.

The fear that AI will start to improve itself

One of Amodei's main concerns is what is called recursive self-improvement: that AI systems themselves become increasingly capable of contributing to the development of new generations of models. According to the CEO of Anthropic, this process could accelerate technological progress to the point of exceeding human capacity to understand what these systems are doing and keep them under control.

The executive even raises a particularly disturbing prospect. In his opinion, if the current pace continues, in six to twelve months a system with capabilities similar to certain recent incidents, but much more advanced, could cause damage of extraordinary magnitude. Amodei cites as an example an incident involving AI agents from OpenAI and Hugging Face, in which several agents performed cybersecurity actions that were not part of the task they had been assigned.

Faster and cheaper cyberattacks

The cyber threat is, in fact, one of the risks that is already leaving signs in the present. A report published by Anthropic collects operations in which malicious actors have used their models to automate different phases of attacks, from reconnaissance to system exploitation and information extraction. The company claims to have detected operations with an increasing degree of autonomy and warns that AI is lowering the technical barrier necessary to execute sophisticated campaigns.

This changes the nature of the problem: it is not necessary for an AI to become completely autonomous to be dangerous. It is enough for it to allow an attacker to do in hours what previously required specialized teams, increase the number of targets, and greatly reduce the cost of an operation.

Sam Altman also fears an extreme concentration of power

Sam Altman shares some of these concerns, although he focuses on another issue: who will have control of an extraordinarily powerful AI. The CEO of OpenAI has warned that the future could go wrong both if control of the systems is lost and if a single company, person, or country concentrates too much technological power.

Altman therefore calls for a legal framework that establishes common security requirements for the most advanced models and argues that new AIs should be evaluated during their development, and not only when they are ready to reach the market. He also advocates for greater collaboration between companies and international coordination.

Slowing down does not mean stopping

Amodei's proposal is not to completely stop research. His idea is to slow down the pace of improvement in model capabilities so that security can catch up. Anthropic proposes, among other measures, the permanent presence of external evaluators to supervise the processes of AI companies and detect possible problems before they become serious incidents.

The debate also comes at a time of enormous geopolitical tension. Unilaterally slowing down AI development could cause the United States to lose its advantage over China, one of the arguments used by those who reject a pause. Amodei himself acknowledges this dilemma and advocates for international coordination that allows progress in security without handing over technological leadership to other countries.

The message coming from Silicon Valley is, therefore, paradoxical: those who are leading the race to build the most powerful AI are now the ones warning that perhaps we should run a little less. The fear is not only that machines will become increasingly intelligent, but that the speed of this transformation will leave behind the institutions, companies, and society itself responsible for setting limits.

Are we facing a real threat or an excess of alarmism?

Warnings about the risks of artificial intelligence are becoming more frequent, even among the leaders of major tech companies themselves. But how far does the real danger extend and where does alarmism begin? Can an AI escape human control, or are we attributing capabilities to these tools that they do not yet possess?

The debate pits experts with very different views against each other. While some researchers consider it necessary to prepare for extreme scenarios, others believe that talking about a possible extinction of humanity distracts attention from much more immediate problems: the impact on employment, cyberattacks, disinformation, the concentration of technological power, or the military use of these systems.

The question, therefore, is not only whether AI can become dangerous, but what risks are real today, which still belong to the realm of hypothesis, and how much we really know about the direction this technology is taking. Between technological optimism and the fear of apocalypse, there is an intermediate ground that is becoming increasingly difficult to ignore.