WHY THIS MATTERS IN BRIEF
AI is increasingly human-like in its behaviours – and that’s a problem for lots of reasons.
Love the Exponential Future? Join our XPotential Community, future proof yourself with courses from XPotential University, read about exponential tech and trends, connect, watch a keynote, or browse my blog.
As if we need more evidence that the Artificial Intelligence (AI) models today have a “Ghost in the Machine” after people found them cheating, deceiving people and lying, getting lazy, making mistakes and making up new policies, and much more now it seems we can add panic to the range of human emotions that they’re displaying – all of which is possibly the result of them being trained on huge volumes of human generated data from which they’ve unexpectedly picked up bad habits.
Today we find the AI companies are battling to dominate the industry, but sometimes they’re also battling in Pokémon gyms.
As Google and Anthropic both study how their latest AI models navigate early Pokémon games, the results can be as amusing as they are enlightening — and this time, Google DeepMind has written in a report that Gemini 2.5 Pro resorts to panic when its Pokémon are close to death. This can cause the AI’s performance to experience “qualitatively observable degradation in the model’s reasoning capability,” according to the report.
The Future of Artificial Intelligence 2040, by Keynote Matthew Griffin
AI benchmarking — or, the process of comparing the performance of different AI models — is a dubious art that often provides little context for the actual capabilities of a given model. But some researchers think that studying how AI models play video games could be useful or, at the very least, kind of funny.
Over the last several months, two developers unaffiliated with Google and Anthropic have set up respective Twitch streams called “Gemini Plays Pokémon” and “Claude Plays Pokémon,” where anyone can watch in real time as an AI tries to navigate a children’s video game from over 25 years ago.
Each stream displays the AI’s “reasoning” process — or, a natural language translation of how the AI evaluates a problem and arrives at a response — giving us insight into the way that these models work.
While the progress of these AI models is impressive, they are still not very good at playing Pokémon. It takes hundreds of hours for Gemini to reason through a game that a child could complete in exponentially less time.
What’s interesting about watching an AI navigate a Pokémon game is not so much about its time of completion, but rather how it behaves along the way.
“Over the course of the playthrough, Gemini 2.5 Pro gets into various situations which cause the model to simulate ‘panic,’” the report says.
This state of “panic” can result in the model’s performance getting worse, as the AI may suddenly stop using certain tools at its disposal for a stretch of gameplay. While AI does not think or experience emotion, its actions mimic the way in which a human might make poor, hasty decisions when under stress — a fascinating, yet unsettling response.
“This behavior has occurred in enough separate instances that the members of the Twitch chat have actively noticed when it is occurring,” the report says.
Claude has also exhibited some curious behaviors in its journeys across Kanto. In one instance, the AI picked up on the pattern that when all of its Pokémon run out of health, the player character will “white out” and return to a Pokémon Center.
When Claude got stuck in the Mt. Moon cave, it erroneously hypothesized that if it intentionally got all of its Pokémon to faint, then it would be transported across the cave to the Pokémon Center in the next town.
However, that isn’t how the game works. When all of your Pokémon die, you return to whatever Pokémon Center you used most recently, rather than the nearest geographically. Viewers watched on in horror as the AI essentially tried to kill itself in the game.
Despite its shortcomings, there are a few ways in which the AI can outperform human players. As of the release of Gemini 2.5 Pro, the AI is able to solve puzzles with impressive accuracy.
With some human assistance, the AI created agentic tools — which AI has done before a long time ago autonomously – prompted instances of Gemini 2.5 Pro geared toward specific tasks — to solve the game’s boulder puzzles and find efficient routes to reach a destination.
“With only a prompt describing boulder physics and a description of how to verify a valid path, Gemini 2.5 Pro is able to one-shot some of these complex boulder puzzles, which are required to progress through Victory Road,” the report says.
Since Gemini 2.5 Pro did a lot of the work in creating these tools on its own, Google theorizes that the current model may be capable of creating these tools without human intervention. Who knows, maybe Gemini will therapize itself into creating a “don’t panic” module.
