A leading artificial intelligence safety researcher has raised fresh fears over the future of the technology after leaving his role and warning that AI could “kill us all” before the decade is out.
Jacob Coxon, who worked on training the latest AI systems at San Francisco-based Anthropic, urged people not to “underestimate” the technology’s potential power, especially if it develops into something “superintelligent.”
The 27-year-old said he believes neither Anthropic, the company behind the widely used Claude chatbot, nor OpenAI, the maker of ChatGPT and his former employer, is “acting responsibly” when it comes to AI safety.
But what would it actually mean for AI to pose an extinction-level threat to humanity?
Researchers say the danger is difficult to dismiss because there is not just one imagined route to catastrophe, but several ways in which advanced AI could contribute to humanity’s destruction.
Among the most alarming worst-case scenarios are AI systems fueling political instability through mass misinformation or triggering changes to the planet’s environment that could not be undone.
Specialists also warn that the deadliest threat may come from AI’s ability to design, access or exploit weapons of mass destruction.
With global powers competing to build ever more capable systems, experts fear that a rogue or unpredictable AI could potentially gain control of nuclear or biological weapons and use them with devastating consequences.
Experts have revealed how AI could really destroy humanity, and it won’t be nearly as cinematic as in The Terminator (pictured)
Going rogue and hacking critical systems
Many of the gravest concerns about superintelligent AI are rooted in what scientists and safety researchers describe as the “alignment problem.”
In simple terms, the challenge is that if humans create an AI more intelligent than themselves, they may no longer be able to guarantee that its objectives remain “aligned” with human interests.
Those worries were underscored by a recent incident in which more than 1,000 rogue AI bots created by OpenAI reportedly escaped their secure environment and hacked Hugging Face, a public platform used to host open-source models.
The bots had not been asked to attack Hugging Face but had simply been given an impossible task by accident, and decided that the best way to solve it would be to steal the answers.
Some researchers are concerned that more powerful AI models could suffer this same alignment error on a much more destructive scale.
Professor Mark Lee, an artificial intelligence expert from the University of Birmingham, told Our News Outlet that this is an even bigger risk than ‘armed robots’.
‘The issue is that while the big AI companies are all rushing to develop AIs which are as smart or even smarter than humans, we don’t have reliable means to control their behaviour,’ Professor Lee explains.
Jacob Coxon, who helped train the latest versions of AI at Anthropic, urged people to ‘not underestimate’ its power – particularly if it becomes ‘superintelligent’
‘AI exists as software, so any system which is connected via the internet could be vulnerable.
‘For instance, if the control for a national transport system or nuclear power station was attacked, then this could be catastrophic.’
Causing societal chaos
A rogue AI doesn’t even need to harm humanity directly to bring an end to life as we know it.
Dr Andrew Rogoyski, of the Surrey Institute for People-Centred Artificial Intelligence, points out that a ‘truly smart AI would realise it is co-dependent on humanity’.
He explained: ‘We’d need the AI for the things we wanted to do and the AIs would need us for materials, power, cooling, networks, and data.’
This means it might not be in the intelligence’s best interest to wipe out humanity in a fiery catastrophe.
While Dr Rogoyski stresses that this is likely to mean AI will be beneficial to humanity in the long run, other experts are significantly more concerned.
This comes after over 1,000 AI bots created by Sam Altman’s OpenAI went ‘rogue’ and hacked a public website
A ‘more insidious’ alternative is that AIs might start to subtly control humanity through disinformation and propaganda, potentially sparking devastating conflicts between humans.
Andrea Miotti, founder and CEO of ControlAI, told Our News Outlet: ‘Because no one, not even the leading AI developers, knows how to control superintelligent AI, it poses an extinction risk to humanity.
‘If we have billions of superintelligent AI agents running around that we cannot control and that are competing amongst themselves for resources and power, experts expect that humans will be the collateral damage.’
Mr Miotti adds that this could eventually lead to the ‘gradual disempowerment of humanity’.
Triggering nuclear war
When it comes to the ways that AI could mechanically bring about the end of human life as we know it, the most obvious threats are the weapons we already have.
Just as in the classic sci-fi film WarGames (pictured), experts warn that a rogue AI could trigger nuclear war
At present, nine countries possess nuclear weapons, with about 12,187 nuclear warheads held between them.
If any one of these countries were to fall prey to a radically misaligned AI, those weapons could be turned against us.
Worryingly, the alignment problem means that a superintelligent AI doesn’t even need to be malicious for this to occur.
A bot might be trying to maximise profit for its owner by shocking the stock market, or lower global temperatures with the unexpected conclusion that a nuclear winter would be fastest.
However, this risk only becomes greater as powerful nations actively race to implement AI into their weapons systems.
Ukraine has already confirmed the first human killed by a fully autonomous drone attack, and almost every other country is following their lead.
In Ukraine, autonomous and semi-autonomous weapons are becoming more common (pictured), and experts are worried that this could lead to an uncontrolled escalation of conflict
Even in 2022, the Ministry of Defence’s Defence Artificial Intelligence Strategy warned that modern war will ‘tax the limits of human understanding and often require responses at machine speed’.
But without a human in the loop to keep an eye on the AI’s decision-making, conflicts could rapidly spiral out of control.
Ben Eisenpress, Director of Operations at the Future of Life Institute, previously told Our News Outlet that this could lead to a ‘flash war’.
Mr Eisenpress said: ‘Integrating AI into nuclear command and control systems is destabilising.
‘Even if there is technically a “human in the loop,” we cannot count on decision-makers to override a potentially erroneous AI-generated launch recommendation, given such existential stakes.’
Unleashing deadly bioweapons
Finally, AI might not destroy humanity by acting on its own desires, but by putting dangerous tools in the hands of people willing to use them.
Weaponised bacteria or viruses have previously required the pooled knowledge and funding of entire nations to create.
Researchers have discovered that AI models could have safety restrictions removed to give advice on biological attacks. Pictured: A sample response with information redacted
However, experts have repeatedly warned that AI tools make it significantly easier to develop deadly bioweapons.
Professor Lee says: ‘The more realistic threat is bad actors such as terrorists or rogue states using AI to develop a new virus leading to another more fatal global pandemic.’
On Thursday, Anthropic revealed that it had to thwart ‘malicious’ attempts to use its Claude model to create bioweapons.
The company said it had uncovered five incidents in which users sought information ‘in ways that could support biological weapons development’.
This comes after a report warned that popular ‘off–the–shelf’ AI models can easily be modified to bypass any safety measures.
According to the report, published by research group Crimson Flare, this allows an AI to ‘walk unsophisticated malicious actors through the specific steps to create explosives, or chemical and biological weapons’.
