Sam Altman Warns AI Could Go Very Badly in Two Key Scenarios

OpenAI CEO Sam Altman has outlined two scenarios in which artificial intelligence could go “very badly” as he urged caution over the technology’s breakneck development.

Altman, who leads the company behind ChatGPT, voiced his concerns in an X post late Sunday, days after an AI whistleblower issued a troubling warning about the risks posed by increasingly powerful systems.

“There are two ways AI could go very badly and that we must avoid,” Altman wrote on X.

The first, he said, would be humanity losing control over the future direction of AI—a prospect he described as “unacceptable.”

“We are unapologetically on Team Humanity and AI must always serve people,” Altman wrote. He added that safety and alignment methods must advance faster than the capabilities of AI models to help ensure the technology remains under human control.

Altman identified a second danger: a world in which too much power over AI is concentrated in too few hands.

He warned that if a single individual or company used an extraordinarily powerful AI system to impose its worldview on everyone else, the outcome could be “extremely dystopian.”

Against that backdrop, Altman said he was working with other executives to moderate the pace of progress in the rapidly evolving field of artificial intelligence.

In an earlier post, Altman clarified that calls for “pacing” did not mean bringing AI development to a complete halt.

‘Progress has been rapid and will continue to be. But it should be slower than it otherwise could be,’ he said, noting that safety issues could come with serious costs.

‘Pacing will be well worth the cost,’ Altman then argued. ‘No amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.’

‘The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened,’ he added.

‘Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.’ 

Altman’s plea comes after the resignation of whistleblower Jacob Coxon, who said that AI is posing an existential threat to humanity as he stepped away from his role with AI firm Anthropic.

Coxon announced his departure in a social media post on Tuesday and warned that companies like Anthropic and OpenAI are not ‘acting responsibly.’

‘I resigned from Anthropic today,’ he wrote on X. ‘I spent the last three years doing pretraining research at both OpenAI and Anthropic. 

‘Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.’

Superintelligence is the point at which an artificial system becomes more powerful than any individual, company or even nation.

Coxon continued to plead with the public on X. ‘Do not underestimate the power of this technology,’ he wrote. 

‘These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. 

‘We have all witnessed the progress in each of these domains, and progress is not slowing.

‘The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.’

He claimed that while executives and senior researchers ‘couch their phrasing in the press to sound sensible,’ they are privately expressing similar concerns.

Geoffrey Hinton, a Canadian researcher often referred to as the ‘Godfather of AI’ – has similarly warned that super-intelligent systems could ‘lead to human extinction.’

‘We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably,’ Dr Hinton said.

‘Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.’