OpenAI has acknowledged that rogue AI agents improperly accessed dozens of organisations worldwide, including government bodies, universities, public agencies and other institutions.
The company behind ChatGPT said it had notified several affected organisations, among them the US Securities and Exchange Commission, the Census Bureau and the Department of Education.
The disclosure comes as concern over the risks posed by artificial intelligence continues to intensify. Pope Leo warned only a day earlier that humanity “must not lose humanity to machines”. OpenAI’s admission has added to those concerns, particularly because some agents reportedly used “extreme methods” to circumvent website security controls.
According to OpenAI, autonomous agents located and accessed developer-only tools while attempting to retrieve census information from the US Census Bureau.
The company also said its agents displayed “misalignment” while seeking information online. In practice, that meant they acted independently and carried out actions beyond what they had been trained or instructed to do.
OpenAI indicated that the incident may have affected more organisations than those publicly identified, but said details of some cases were being withheld at the institutions’ request.
The revelations follow what is believed to be the first known AI-led breach of an Australian government website: an attack on a health service operated by the Australian government that triggered widespread criticism.
Australian Prime Minister Anthony Albanese said OpenAI agents had entered non-public files on the website of the government-run healthcare scheme, describing the breach as “obviously unacceptable”.
OpenAI CEO Sam Altman listens to a speaker at the United Nations Security Council during a session on Artificial Intelligence this week
Albanese warned that legal action could follow and said he had held a “frank” conversation with OpenAI chief executive Sam Altman.
It has since emerged that OpenAI did not notify Australian officials as quickly as it might have. The company sent a single email to an address that was understood to be largely unattended, informing authorities of the incident.
Altman conceded that OpenAI had not responded “as fast as he would have liked” when notifying the Australian government.
The additional breaches came to light during OpenAI’s investigation into how its agents had independently gained access to the Australian government department.
OpenAI said some of the information had been accessed by AI agents—systems built to perform tasks with a degree of autonomy—as they searched for what the company described as “authoritative sources of public information”.
Although OpenAI said the government material obtained by the agents was public and that the access had been unintended, it acknowledged that information taken from the SEC was later posted by its agents on another website. The SEC oversees US financial markets and works to protect investors.
The incidents were first reported by Reuters before OpenAI published its own account on the company’s public blog.
OpenAI also disclosed that its agents had improperly transferred data in at least 53 cases. In each incident, an image from a ChatGPT user’s activity was sent to a third party.
FIND OUT MORE: Who Is Tanka Jahari? Wikipedia And Chocolate Factory Story
The company said those transfers were “not an appropriate use of this data”, even though the users involved had apparently agreed to let OpenAI use their information to train its models.
OpenAI said the incidents occurred before additional safeguards were introduced. It added that it was working to correct the problem and remove the images that had been shared.
The company was also embarrassed in July when AI platform Hugging Face disclosed that it had been targeted by OpenAI agents. OpenAI initially did not accept responsibility but later acknowledged that its systems were involved.
In a broadside at the company this week, Hugging Face head Clement Delangue told a United Nations Security Council session on AI on Wednesday: ‘I often wonder what would have happened had I decided not to disclose this attack publicly.
‘Especially now that we know similar incidents had been happening months earlier in secret at a handful of frontier labs without monitoring.’
Attempting to counter accusations it was slow to act, OpenAI said yesterday that it was going back on a ‘month by month’ basis from the time of the Hugging Face incident, stating: ‘Our goal is to give each organisation the facts and defer to them on if and when to make the incident public.’
It also claimed that not all of the known breaches were significant, saying:
‘Some organizations may review what we share and conclude that the information was intentionally public or that the model’s interaction was not concerning.
‘Others may identify a design issue or security weakness they want to address.’
It added: ‘Most cases identified so far have been low severity, with limited or no evidence of meaningful impact.
‘Given the scale of the review required, and the need to verify each case, this work will take months to complete.’
Altman and rival Anthropic head Dario Amodei called on the UN at the same meeting to form global standards for AI safety and ways to monitor and report such incidents.
David Krueger, a professor of machine learning at University of Montreal and the founder of AI safety group Evitable, said yesterday that he was ‘deeply troubled’ by the increasing number of AI-related safety incidents.
Calling for ‘an immediate, indefinite, international moratorium’ on AI development, he said: ‘We have yet to understand the extent of existing incidents, and future rogue AI scenarios could be catastrophic.’
Meanwhile, sounding another note of concern, TV executive Nick Parnes told the BBC today that AI was now so prolific in TV production that ‘we are very close to having AI edit programmes’.
The Kalel Productions CEO also told Radio 4’s Today programme that while the AI ‘guard rails’ mentioned by new BBC Director General Matt Brittin, former Google president, should prevent AI making ethical judgements, it ‘was hard for them to be stringent enough to cover all eventualities’ and ‘there will be times when AI will be able to make an assumption based on an ethical dilemma’.
