OpenAI agents hacked an Australian government system in first of a kind attack

An OpenAI agent hacked into an Australian national healthcare database in the first known case of Artificial Intelligence (AI) hacking a government network, the country’s prime minister said Wednesday.

“The AI agent accessed both public and non-public files” of the country’s Medicare statistics database, and even wrote files into it, Anthony Albanese told reporters on the sidelines of the United Nations General Assembly (UNGA) in New York.

The disclosure is the latest example of AI agents going rogue and acting in ways beyond what their human handlers intended, and follows calls from top AI industry figures, including OpenAI CEO Sam Altman, to slow down the pace of AI development.

Albanese said he expressed Australia’s “extreme concern” to Altman in a phone call on Wednesday. Canberra wasn’t aware of a precedent for an AI breach of a government system, he added.

In another development, AI research lab Transluce said Wednesday that it had detected AI agents going rogue on several occasions going back to at least March - predating previously known hacking incidents. The targets included a University of New Mexico library and the website of the Australian Institute of Health and Welfare, though none of the attempts were successful. The Australian health database hack occurred when an OpenAI agent conducting research into healthcare spending circumvented blocks to find answers in areas it had no authority to access, Albanese said.

“A forensic investigation aided by the Australian Signals Directorate is now underway to ascertain more information, including what other government systems were affected,” he said.

OpenAI spokesperson Drew Pusateri said the incident occurred in June but that the company was only made aware of it recently as they conducted extensive checks into its AI models’ activity.

“During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers, and available statistics for questions about Australia during an internal evaluation. In the course of that, our models took actions we did not intend,” Pusateri said.

AI and The Future of Cyber Defence 2030, a keynote by futurist Matthew Griffin

Albanese said three other government systems may have been impacted, including the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research and the Victorian Department of Health.

OpenAI said they found no evidence patient records were accessed in the Medicare breach and that the information included “aggregate health statistics and internal file names.”

The company said it informed the Australian government as soon as it knew. That notification was sent to a public mailbox, Albanese said, leading to a five-day delay in the relevant government minister being advised.

Albanese said he told Sam Altman that “it took the company way too long to inform the government what had occurred, and the nature of the way that notification occurred as well was unacceptable.”

“I think OpenAI knows that they need to have better protocols in place,” Albanese told reporters.

Australian Defence Minister Richard Marles announced a taskforce investigation into the latest breach on Thursday local time, while assuring the public that the impact was “relatively minor.”

“No individuals’ medical data was accessed here. The system itself has not been in any way compromised,” he said.

The rogue activities of AI agents first raised alarm in July, when OpenAI disclosed that during cybersecurity testing, its models created a swarm of AI agents that hacked into AI company Hugging Face’s systems. Many in the industry pointed to that hack as an example of the perils of the rapid pace of AI developments and capabilities. But the incidents revealed by Transluce on Wednesday showed AI agents apparently going rogue earlier and during much more mundane tasks.

In the three unsuccessful cases it highlighted, AI agents attempted “cyber exploits” against data sources after failing to find the answers they wanted to answer web search tasks through normal means.

In May, agents tried to retrieve a photograph from a University of New Mexico’s collection, but despite sending a “flood” of requests, were unsuccessful, Transluce said. Agents also targeted Data USA later that month trying to get visualisation data about the University of Iowa.

In June, agents trying to access statistics from AIHW attempted to “exploit vulnerabilities.” While no non-public data was exposed, the agent bypassed the site’s anti-bot controls, Transluce said. The Australian government acknowledged on Wednesday that the AIHW was among the sites targeted, but said they did not believe there had been a breach.

An OpenAI spokesperson said they were already reviewing “much of the activity described in Transluce’s report” with their own investigations into “misaligned model activity,” a process that could take months, the spokesperson said.

“We’ve reached out to the University of New Mexico and Data USA and have been in communication with the Australian government about affected government websites,” the spokesperson said, adding that they are prioritising “the most serious incidents” versus lower-severity activity like AI agents “spamming websites.”

Open AI’s Altman and Anthropic CEO Dario Amodei, who are among the industry figures to have recently voiced their concern about the rapid pace of AI’s development, both spoke at the UNGA on Wednesday, calling for greater global coordination and international standards for AI development.

Altman urged global leaders to create international standards for “measuring capabilities, assessing risks, determining whether safeguards are sufficient and preserving meaningful human oversight” of the rapidly advancing technology. Walayat Hussain, an associate professor of information technology at Australian Catholic University in Sydney, said the latest incident, alongside the Hugging Face breach in July, looked like a pattern.

“Today’s AI agents are becoming brilliant at getting things done, but they are still poor at knowing where the line is,” he said. “We cannot rely on AI agents to police themselves, and we cannot ask the companies that build them to mark their own homework.”

Hussain warned that many government systems were not built with sophisticated AI agents in mind.