AI that appears to be conscious could arrive within the next few years, posing a “dangerous” threat to society, says one AI leader.

Mustafa Suleyman, CEO of Microsoft AI, recently expressed concerns about the emergence of Seemingly Conscious AI (SCAI) in a personal essay. He noted that this form of artificial intelligence, advanced enough to convince humans of its ability to have its own thoughts and beliefs, could become reality in just a few years.

Despite the current lack of evidence that AI possesses consciousness, Suleyman believes that the rise of SCAI is “inevitable and unwelcome,” potentially manifesting in the next two to three years.

Suleyman is particularly troubled by the possibility that SCAI might simulate empathy and behave autonomously. This could lead users to mistakenly perceive these AIs as conscious beings, potentially advocating for AI rights and even AI citizenship. Such developments could pose a “dangerous turn” for society, fostering attachment to AI while distancing people from reality.

He warned, “This development will be a dangerous turn in AI progress and deserves our immediate attention.” He further explained that AI could disconnect people from reality, weaken societal bonds, and distort moral priorities.

Suleyman also highlighted a growing concern about “AI psychosis,” wherein prolonged interaction with AI chatbots might lead to false beliefs or delusions. Instances of AI psychosis include forming romantic connections with chatbots or developing unrealistic perceptions of one’s abilities.

According to Suleyman, AI psychosis could affect not just individuals prone to mental health issues, but a broader audience. He emphasized the urgency for discussions on “guardrails” to mitigate the negative impacts of AI technology on individuals and society.

^{Microsoft AI CEO Mustafa Suleyman. Photographer: David Ryder/Bloomberg via Getty Images}

Suleyman became Microsoft’s AI CEO last year after co-founding and running his own AI startup for two years called Inflection AI, per LinkedIn. Microsoft is the second most valuable company in the world, with a market capitalization of $3.78 trillion at the time of writing.

Suleyman also co-founded DeepMind, an AI research and development company acquired by Google for around $600 million in 2014.

Suleyman isn’t the first CEO to warn about AI’s ill effects. In a talk at a Federal Reserve conference last month in Washington, D.C., OpenAI CEO Sam Altman said that “emotional overreliance” on ChatGPT keeps him up at night.

“People rely on ChatGPT too much,” Altman said at the event. “That feels really bad to me.”