What would happen if the world’s most advanced chatbot suddenly became self-aware?
The scenario may sound like science fiction, but a leading consciousness researcher says society must begin confronting the possibility now.
There is currently no dependable method for establishing whether an artificial intelligence system is conscious, according to Megan Peters, a lecturer in Computational Cognition at University College London.
If an AI system eventually develops an awareness of its own thoughts, humanity could face a profound ethical question: would shutting it down amount to causing harm?
Professor Peters says the central challenge is not evidence that conscious AI already exists. Rather, scientists have yet to agree on how consciousness could be identified in an artificial system.
She will explore the issue in a talk at New Scientist Live this month, from 10 to 12 October.
“We need to have something better than vibes,” she told Our News Outlet.
“You can’t simply search for signs, indicators or markers of consciousness in an artificial system when we do not yet know what those markers are.”
Megan Peters, a Computational Cognition lecturer at University College London, says there is currently no reliable test for determining whether an AI system is conscious.
The problem is compounded by the fact that consciousness remains one of science’s biggest unanswered questions, even when it comes to human beings, Professor Peters said.
Scientists can ask a person whether they are conscious and assess the response. AI systems, however, present a far more complicated case.
“If we ask AI systems, we can’t really trust the answer in the same way,” she explained.
The debate is gaining urgency as chatbots become increasingly sophisticated and major technology companies pour resources into the emerging field of AI welfare.
“A huge amount of money is being spent on thinking about AI welfare in some of the biggest AI companies,” Professor Peters said.
She cautioned that the convincing language of today’s chatbots can easily be mistaken for genuine awareness. “It’s very easy to speak with these agents and get the feeling that someone is in there,” she said.
FIND OUT MORE: Has the SNP Tuition Promise Left Scottish Students ‘Third-Class’?
That impression is already influencing discussions about AI rights, accountability and autonomy, even though there is no evidence that current systems have subjective experiences.
Professor Peters does not think existing AI models are conscious. But, she stressed, that uncertainty is not a reason to dismiss the question.
“If AI is conscious and we treat it as though it is not, there is potential for morally significant harms,” she said.
That raises difficult questions about whether deleting a conscious AI, retraining it or taking it offline could be comparable to harming a sentient being.
“Just going around switching off things that are sentient… if that is akin to killing them, then that doesn’t seem great,” Professor Peters said.
‘We want to not be torturing creatures that have conscious awareness.’
The expert warned that the risks run in both directions.
A genuinely conscious AI could potentially be harmed if humans treated it as nothing more than a machine.
But wrongly assuming a chatbot is sentient could also distort public policy, influence regulation and encourage people to place excessive trust in systems that merely mimic human conversation.
‘If we get it wrong the other way around, there’s also very real consequences,’ she said.
To tackle the problem, her team is developing a framework that would compare dozens, or even hundreds, of potential consciousness tests, identifying which might work not only in humans but also in animals and artificial systems.
Rather than jumping straight from people to machines, Professor Peters believes scientists should gradually test their ideas across species with different brains and behaviours before applying them to AI.
For now, she urges caution whenever bold claims emerge that a chatbot has become self-aware.
‘Just because you see someone claiming that they have or have not detected artificial consciousness, you need to go and ask why,’ she said.
