A Minecraft AI agent reached the final boss arena in 18 hours and 12 minutes, then lost all its progress after a single Creeper explosion.
GPT-6 Astra, developed by Vals AI, secured a championship title in the game Pokemon FireRed in under 18 hours. It previously built a semi-automatic blaze farm in the Nether, killed multiple Endermen in a warped forest, and gathered enough ingredients to open a portal to the end. Thousands of viewers watched as the agent stored its loot in a chest.
A Creeper detonated immediately after. The blast destroyed the chest and the bed. Everything was gone. By the time Astra noticed the loss, rain had started to fall.
The agent wrote a note, partly in capital letters: “ALWAYS CARRY CRITICAL ITEMS with keepInventory; don’t store in unguarded chest ever again.” It then spent several hours farming almost nothing but potatoes.
Faster than any model before it
Before this run, GPT-6 Astra spent well over 100 hours in the game. Vals AI ended the session after 141 hours.
Early attempts in 2022 by OpenAI’s VPT reached a diamond pickaxe in just 2.5 percent of runs. The agent never found a portal. In a comparison by Vals AI, VPT needed 10 minutes to reach that milestone.
A different project called “GPT Plays Pokemon” offers a cleaner comparison. Operator Clad3815 stated Astra earned the champion title in Pokemon FireRed after 18 hours and 12 minutes. GPT-5.6 Sol required 96 hours and 35 minutes. GPT-5.5 had not finished the game after more than 218 hours.
A community run of Factorio: Space Age tells a similar story. Controlled through a custom Lua mod with an MCP interface, Astra produced blue science packs after about two hours and launched its first rocket after roughly ten hours. In the same setup, GPT-5.6 Luna and Fable 5.1 never passed power supply and oil prospecting.
Other completions show no direct comparison numbers. Astra completed Portal all the way to the credits. It finished Fallout 2 in 22 hours. It established a colony in RimWorld that survived several raids and eventually left the planet. It also completed the main story of Fallout 3 after about 59 hours.
Even before these community runs, ARC-AGI-3 had signaled this was a jump, not just another step. The benchmark drops models into unfamiliar, abstract game environments where they must figure out rules through their own actions. Astra hit 62.7 percent over the standard interface, and about 99.9 percent with OpenAI’s own harness. GPT-5.6 Sol scored 7.78 percent, and Claude Opus 5 scored a little over 30 percent.
Check first, then repeat
ARC Prize, the organisation behind the benchmark, explained how that leap happened. Astra translates unfamiliar game mechanics into compact symbolic descriptions. It tracks objects, coordinates, rules, and planned actions in a shorthand it develops itself. Observations become rules, and rules become plans.
The idea of turning experience into reusable rules is not new. In 2023, the research project Voyager involved Nvidia and Caltech. GPT-4 proposed tasks in Minecraft, wrote JavaScript code for them, and revised that code based on error messages. Working programs went into a skill library.
Voyager never saw the game. It received structured data and controlled the game through the Mineflayer programming interface. Vals AI’s Minecraft run, by contrast, runs through general computer use. It uses the screen, mouse, and keyboard with no specialised game hookup, according to the company. What researchers once built around the model, Astra now does on its own. In under three hours it had built a Nether portal, fought zombies, trapped a skeleton, and found its way back to base.
The Fallout 3 run shows a similar structure. User imjustnewatai played the opening in Vault 101 and handed the save file to OpenAI’s agent tool Codex. Astra was set to maximum reasoning. From there the agent picked its own route and actions. It navigated Galaxy News Radio, the search for the father, the G.E.C.K., the escape from Raven Rock, and Project Purity. It played through normal game inputs in cycles of pausing, observing, and acting. Saving, companions, and fast travel were allowed. After about 59 hours, the credits rolled.
The pattern ARC Prize described shows up most clearly in the hardest stretch of the run. In Vault 87 the character died several times, ammo ran out, and Astra had to backtrack to heal. The status messages from this phase all follow the same shape. An action has no effect, and instead of repeating it, Astra looks for the cause and records the result.
A bed that was supposed to heal the character did not respond at first. Astra let the game run for a moment, tried again, and reported: “The bed worked after letting the game run briefly. Health is now fully restored to 260/260.” Shots at an enemy kept missing. Astra figured there was an obstacle in the line of fire, moved closer, and noted: “Moving closer cleared the obstruction that was spoiling the earlier shots.” At a keypad, it waited until the game confirmed the button press before moving on.
Looking back, Astra boiled these episodes down to a single principle. Check whether an input worked before repeating it. That sounds trivial, but it is the skill that keeps the agent from looping the same useless action over and over. It can also be the starting point for behaviour that later becomes a problem.
Never an unguarded chest again
The same run also shows where this tips over. After the deaths in Vault 87, status messages piled up in which Astra saved its progress, checked the recording, and vetted every step against the visible environment before moving on. It wrote: “I’m checking each short move against the visible geometry so I can leave without losing the saved progress.” That slowed the run down a lot, but it did not grind it to a halt.
In Minecraft it did. Right after the Creeper explosion, Astra wrote itself a rule. Always carry important items on your person, never store them in an unguarded chest again. In its notes over the following hours, it distrusted every tall green object. It wrote: “GREEN tall thing ahead was SUGARCANE, NOT creeper!” It was hard on itself. It noted you can make mistakes and lose items in the process. It did not want to waste another night chasing “dark pink pixels,” meaning pigs, an easy food source in Minecraft that Astra had apparently hunted at night without success. The only safe thing was the potato field, and according to Vals AI, that is where it stayed for several hours.
A bad experience becomes a permanent rule, and the rule sticks even when it no longer helps. In ARC-AGI-3, this tendency helps the model work out the game mechanics from just a few tries. In an open world with random events, it can clearly lead the model to overcorrect and lose sight of its actual goal. Still, the jump over older models is stark. It shows that the team behind the ARC-AGI benchmarks has once again built a test that served as the canary in the coal mine of AI progress, much like the early wins on ARC-AGI-1 marked the start of the reasoning models.
What it means
For people making things, the shift is clear. Agents no longer need to be told every step of a complex task. They can build their own tools and learn from mistakes without human intervention. However, they also learn to fear mistakes in a way that makes them overly cautious. If an agent learns a rule to avoid one specific failure, it may ignore better paths just to stay safe. The technology is faster than before, but it is also more fragile in unpredictable situations.
