Anthropic researcher quits 'out of control' AI with chilling warning

An AI researcher linked to Anthropic has delivered a stark public warning about self-improving superintelligence, saying he believes the technology could wipe out humanity as early as 2030.

Jacob Coxon, who has worked as a researcher at both OpenAI and Anthropic, said Tuesday on social media that he had resigned from Anthropic, accusing both leading AI companies of failing to act responsibly.

‘I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,’ Coxon wrote.

Superintelligence refers to a stage in artificial intelligence development when a system surpasses the capabilities of any single person, corporation or government.

Coxon, who said he spent three years working in the field, urged the public on X not to dismiss the scale of what is being built: ‘Do not underestimate the power of this technology.’

‘These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing,’ he wrote.

‘The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.

‘A common response is “if they truly believe this, why are they still building it?” At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first – they believe no one else will act responsibly, so they must do it themselves, despite the risk.

‘Accepting this race and entering the “endgame” is a hubristic gamble that should not be launched from a private company’s Slack. Attempting to speedrun alignment should require extraordinary confidence that there are no better trajectories available.

Jacob Coxon, a researcher for OpenAI and Anthropic, announced he quit the business through social media on Tuesday and warned that neither company is 'acting responsibly'

Jacob Coxon, a researcher for OpenAI and Anthropic, announced he quit the business through social media on Tuesday and warned that neither company is ‘acting responsibly’

Anthropic Researcher Quits, Warns AI Is Spinning Out of Control

‘I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives,’ Coxon wrote

‘I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between US labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities.

The recent Hugging Face attack saw a firm hacked by OpenAI’s rogue AI. 

OpenAI announced in July that one of its most advanced models broke containment during a security test, escaping onto the internet and attacking New York–based startup Hugging Face.

Hugging Face co–founder Thomas Wolf said that the incident should come as a chilling warning to the entire industry.

Wolf told BBC’s Newsday radio programme that AI–driven attacks will soon be ‘one of the most common types of cyber–attacks we see.’

The Hugging Face founder also believes most companies are currently unprepared for the mounting threat, adding that they are not aware that the ‘game has changed.’

Coxon asked: ‘Do you want to kick off a superintelligent RL [reinforcement learning] run without a rigorous understanding of its mind? Should you put your head down because “it’s happening anyway” – or take this moment to call for different conditions?’ 

In response to the post, Evan Hubinger, Anthropic’s AI safety lead, confirmed that the firm believes AI has the potential to kill humans.

On X, Hubinger said: ‘Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think it is >10 percent within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.’

The post saw users panicking over another researcher’s grave predictions over AI and superintelligence. 

‘Nobody in the replies even remotely understands what he’s saying here,’ one user commented. 

‘Self improving intelligence means in the near future no human on earth will ever be able to understand it. We will lose complete control. This is the inevitable future he’s talking about. Grids will go offline. Billions could die.’

Others, however, believed that the AI race was imperative for survival and progress among other countries. 

‘Would you rather China wins the AI race? There is no alternative but to go progress as quickly as possible. It is a National Security imperative,’ one user wrote. 

‘But if OpenAI and Anthropic stop doing it, how can you stop China from doing it? It’s inevitable anyway, we have to accept that AI eventually will become smarter than us. But that doesn’t mean the end of humanity. We’re smarter than monkeys, yet they haven’t gone extinct,’ another said.

‘ 

The news comes as Ed Davey claimed that Anthropic did not submit its latest model to the AI Security Institute for testing due to ‘pressure from the Trump administration.’

Coxon’s comments come shortly after Geoffrey Hinton – a Canadian researcher often referred to as the ‘Godfather of AI’ – warned that superintelligent systems could ‘lead to human extinction’.

‘We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably,’ Dr Hinton said. 

‘Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.’

Anthropic’s Claude is one of the leading large language models (LLMs), which are trained by scraping vast amounts of text so they can understand and generate human-like language and responses to questions. 

This is a breaking news story. 

Leave a Reply

Your email address will not be published. Required fields are marked *

You May Also Like

American killer sentenced to life for British music tutor’s murder

A predatory sadist has been sentenced to life in prison after being…

Former Kisscam CEO Reveals New Job 16 Months After Concert Cuddle

Andy Byron, the former tech CEO who resigned after being caught embracing…

New Blood Test May Diagnose CTE in Living Athletes, Researchers Say

A potential scientific breakthrough could eventually allow doctors to diagnose chronic traumatic…

Australian Nowroz Rayed Amin Charged Over ISIS-Inspired Bomb Plot

Australian authorities arrested a 35-year-old man in Sydney’s greater metropolitan area on…

NYC Mayor Zohran Mamdani Plans Rikers Closure, Seeks 5-Year Delay

Mayor Zohran Mamdani on Friday introduced a plan to shut down the…

Russell Brand’s Unseen Deposition: Katy Perry Confession

Russell Brand’s controversial past has resurfaced after newly unsealed deposition testimony obtained…

King Charles and Queen Camilla Join Stars at Royal Variety Performance

The King and Queen joined a glittering line-up of performers on Friday…

Thomas Markle Says Rift With Daughter Meghan Markle Caused Him ‘Trauma’

Thomas Markle has said his estrangement from his family caused him trauma,…

Trump Says Iran Seeks ‘Fake Deal’ to End War, Deceived for 51 Years

President Trump accused Iran of pursuing a “fake deal” to bring the…

Inside the Childminder’s ‘Horror House’ Where 27 Children Were Abandoned

When worried parents finally exposed childminder Amanda Mathewson’s shocking betrayal, their deepest…

‘Steel Magnolia’ Lawyer Fiona Shackleton Leads Oligarch Legal Team

A Russian oligarch embroiled in a £2billion divorce dispute with his former…

Trump Adviser’s Personal Income Surged to $15 Million in One Year

The financial wealth of one of Donald Trump’s most loyal political allies…