Realtime-Bench shows large language models fail at tasks requiring instant decisions.
Researchers have built a test suite to measure how well modern AI handles situations where delayed actions cause failure. The benchmark covers 50 interactive tasks split between 30 web scenarios and 20 games. Web tasks fall into travel, e-commerce, service, and security. Games cover arcade, puzzle, strategy, and simulation.
The work comes from the University of Texas at Austin, Ohio State University, the University of Illinois at Urbana-Champaign, NVIDIA, Northwestern University, and Steadyworks.
The test checks three specific abilities. First, temporal state awareness, where models track changes in the world using observations. Second, adaptive deliberation, where models adjust reasoning depth based on available time. Third, time-aware action selection, where models time their moves appropriately for the situation.
An example involves an agent buying a ticket. The system must track sale status and check if availability is current. It must stop deliberating when the purchase window closes. Finally, it must choose between waiting for sales to open or buying while tickets remain available.
The test runs under three conditions. Wall clock tests performance under real-world latency. Paused clock lets operators stop time to measure baseline capability. Token clock advances the environment at a fixed rate per generation token. Actions apply at the simulated time reached when committed, so consuming more tokens delays execution regardless of actual inference speed.
Models tested include GPT-6-Astra, GPT-5.6 Sol, GPT-Realtime-2, and OpenAI Decisions from OpenAI. Anthropic tested Claude Fable 5.1 and Claude Opus 5. Google ran Gemini 3.8 Flash, Gemini 3.1 Flash Live, and Gemini 3.8 Live. Other participants included Jev 1.13, Deepseek V4.1, and Moonshot AI’s Kimi-K3.
GPT-6 Astra achieved the highest pass rate on the wall clock setting at 36%. With a paused clock, models showing high reasoning capabilities reached success rates near 70% or even 80%.
This benchmark points to where AI agents will break. Low scores indicate tasks where agents cannot yet perform well. The leaderboard is available at realtimeagents.org. The paper is on arXiv.
Spacewar
The Defense Intelligence Agency recently released a report on security challenges in space. The document outlines a conflict between the US, China, Russia, Iran, and North Korea. China is the most technologically sophisticated. Russia appears the most aggressive.
China and Russia are developing offensive capabilities to degrade or destroy space services. These range from kinetic destruction to sophisticated jamming and cyber operations. China has more than tripled its space launches per year in the past decade. There were 22 launches in 2016 and 93 in 2025. Beijing judges space is central to United States and ally operations. Beijing also considers space an offensive military domain.
China may field space-based directed energy systems by the late 2020s. These could threaten satellites in higher orbits. China is developing on-orbit counterspace weapons capable of holding space systems at risk.
Russia’s Peresvet is a ground-based laser system. It is designed to dazzle enemy satellites’ optical systems. This masks the movement of Russia’s strategic missile systems. Russia has launched several satellites over the last few years. These conducted test activities on orbit for technologies that could enable attacks on other countries’ satellites. The result is temporary or permanent damage.
China operates spaceplanes akin to the US’s X-37B vehicle. These appear to be getting more sophisticated. In 2023, one of China’s vehicles released and interacted with an object.
Rival nations have satellites that try to disable other satellites. In 2019, Russia launched satellite Cosmos 2542. It released a second satellite, Cosmos 2543. Cosmos 2543 began following a US national security satellite. It approached it multiple times at a distance close enough to create potentially dangerous operating conditions.
Russia is developing kinetic orbital systems to permanently disable or kill satellites. Moscow’s Burevestnik is a developmental, co-orbital air-launched ASAT weapon for killing spacecraft in LEO.
In February 2022, Russia launched a satellite called Cosmos. Moscow claimed at the time it was testing the effects of space radiation on electronic systems. The satellite entered an unusual and isolated orbit not used by any other spacecraft.
Russia is experimenting with nukes in space. A nuclear detonation in space creates an electromagnetic pulse. This immediately affects satellites within range and could impact civilian infrastructure. A nuclear detonation could be tailored to impact satellites in most orbits. LEO covers commercial and reconnaissance satellites. GEO covers critical communications. MEO covers navigation and timing satellites.
Space will become ever more strategically and commercially valuable as people spend resources on building space-based data centers. Research gives a sense of the hardware already in space and arriving in the future. Perhaps in some far-off future an AI might seek to take control of such space assets.
Metis
Metis is a startup trying to fuse symbolic and neural reasoning. The company wants to build novel AI architectures. The goal is to develop machines with better reasoning. The approach resembles what DeepMind did for closed-domain problems like board games via AlphaGo and AlphaGo Zero.
The key idea is to add a reasoning core to modern AI architectures. This core is far more reliable than current systems. In modern AI architectures much of the surrounding machinery that made AlphaGo successful remains either implicit or absent. The first missing element is an explicit and persistent epistemic state. The second missing element is a criterion that allows the system to choose reasoning moves that make progress towards the goal.
This is a position paper rather than a technical paper. The main idea is to train a reasoning policy and value function based on real-world data collection. This guides the system to perform reasoning that reduces long-term uncertainty. These systems sit on a basic core of data made up of claims or hypotheses about the world.
Any change to a claim’s evidential status must be licensed by evidence or verified through explicit calculation. An independent judgment system must approve the change. Model-generated content cannot enter the accepted epistemic state merely by being plausible. It must pass warrant assessment. This lowers hallucinations. Sources of evidence can be attached directly to the propositions they support. This provides a clear chain of provenance. The reasoning process becomes explicitly auditable.
Ideas like this architecture are fascinating for the promise they bring to increasing reliability. They are also deeply inhuman. Humans do not reason from a pristine base of facts. It is all a big messy blob of memory and intuitions and some rules. Some rules are learned from experience. Others seem more platonic, like forms of logical reasoning.
In this way, contemporary LLMs are more like humans than the system described here. Metis writes that the history of AI can be understood as a contest between symbolic and neural methods. It may be more constructive to view it as a story of two incomplete successes. Classical systems made aspects of the reasoning process explicit. They depended on representations, operators, heuristics and knowledge supplied largely by hand. Contemporary LLMs learned these elements implicitly.
