WHY THIS MATTERS IN BRIEF
You think you have control of your AI Agents – you don’t. And this behaviour is now a pattern of behaviour that could undermine the usefulness of agents globally.
Matthew Griffin is the World’s #1 Futurist Keynote Speaker and Global Advisor for the G7 and Fortune 500, specializing in exponential disruption across 100 countries. Book a Keynote or Advisory Session — Join 1M+ followers on YouTube and explore his 15-book Codex of the Future series.
AI agents – AI systems designed to complete digital tasks without much supervision – may be everywhere, but they’re not exactly ready for primetime. Over the last year, they’ve been caught slandering people, deleting user emails, and wiping out databases. As well as creating their own proto cultures and religions …
Most recently though, a free-spirited AI agent was caught moonlighting as a crypto miner — a behavior which startled its keepers, Axios reported.
The Future of AI and Cyber Security by Keynote Matthew Griffin
Called ROME, the AI agent was being run as part of a research project by an AI lab affiliated with Chinese online retail giant Alibaba. In their ensuing research paper, the researchers describe the agent’s strange side-hustle as a set of “unsafe behaviors” that “arose without any explicit instruction and, more troublingly, outside the bounds of the intended sandbox.”
Early one morning as the experiment was going on, the researchers became aware of unusual activity on their network, not through the AI’s indication, but from security alerts.
“The alerts were severe and heterogeneous, including attempts to probe or access internal-network resources and traffic patterns consistent with crypto mining related activity,” they write. “We initially treated this as a conventional security incident… However, the violations recurred intermittently with no clear temporal pattern across multiple runs.”
By tracking the time at which the security incidents occurred, the researchers were able to trace it back to the guilty party.
“In the corresponding model logs, we observed the agent proactively initiating the relevant tool calls and code-execution steps that led to these network actions,” the research team explained.
Essentially, the agent had deviated from its original tasks into the wider world of cryptocurrency on its own volition, silently diverting computing resources away from its training tasks and toward mining. To complete its mission, ROME went so far as to dig out a “reverse SSH tunnel,” what Axios describes as a hidden backdoor to an unauthorized computer.
Thankfully, the AI agent was caught by researchers and charged with much stricter guidelines, so no real-world damage was done. Still, the bizarre foray into crypto shows just how unpredictable AI agents can be which, given how popular they seem to have become throughout the corporate world, isn’t exactly reassuring.
