Skyfall AI has launched MORPHEUS, a persistent enterprise simulation benchmark designed to test continual reinforcement learning in environments that do not reset. Unlike standard benchmarks which wipe the world state after every episode, this platform keeps history intact to force agents to adapt to changing conditions.
What is MORPHEUS?
The system is built on the Big World Hypothesis. This theory suggests that real-world complexity exceeds what any single agent can fully represent. Consequently, the environment appears non-stationary even when the underlying rules remain fixed.
MORPHEUS enforces continual learning by requiring three specific conditions. First, decisions made in the past must compound into future dynamics. Second, no fixed policy can remain optimal forever. Third, no single optimal policy exists for the entire duration of the simulation.
Each environment functions as a self-contained TypeScript plugin. These exports include Operational Descriptors, a simulation scheduler, seed data, and documentation. An Operational Descriptor defines the step-by-step execution plan for a specific capability. Agents interact through a capability API, where every call triggers an OD execution.
How the Platform Works
Non-stationarity is driven by two engines. A failure injection engine inserts typed disruptions between Operational Descriptor steps. It draws from eleven failure types, including missing data, dependency failures, and rate limits. These run at four preset rates: light (5%), realistic (8%), moderate (15%), and aggressive (30%).
An asynchronous configuration shift controller changes failure presets and demand levels at fixed timestamps. This controller operates independently of the training loop. As a result, shifts never align with gradient updates. This prevents the agent from using update periodicity as a proxy clock.
Rewards are generated from three operational verifiers logged natively by the platform. These include failure event signals, financial ledger status, and resource throughput. The composite reward combines these factors. Default weights are set at 0.5 for failures and 0.25 for ledger status and throughput.
Under upper-bound assumptions involving zero failures, minimum cost, and full throughput, the bound per configuration equals 0.50.
Policy Initialisation
Because the action space is large, training reinforcement learning from scratch is impractical. Therefore, MORPHEUS uses a two-stage pipeline. A frontier model, Gemini 3.1 pro, collects trajectories using the ReAct framework. These traces then fine-tune Qwen3-14B via supervised fine-tuning.
Every RL run starts from this shared fine-tuned checkpoint. This isolates continual learning behaviour from basic operational competence. All baselines use Proximal Policy Optimisation as the base optimizer for online post-training.
The Six-Metric Evaluation Protocol
Cumulative reward alone does not provide enough detail. A scalar sum hides performance across a non-stationary horizon. The research team proposes six metrics instead. These are per-configuration reward, adaptation speed, forgetting, recovery time, stability, and performance gap.
Adaptation speed is the headline metric. It counts the steps required until the running-average reward reaches half the upper bound. Two supplementary diagnostics also track relative adaptation advantage and plasticity via effective rank.
Baseline Results
Using this protocol, the team tests four algorithm families from the shared fine-tuned checkpoint. Two tasks are defined. Task 1 involves dynamic resource allocation under structured drift. Task 2 involves scheduling under drift with delayed effects.
Across these results, no single family dominates. On the outbound Task 1, EWC leads on reward while LCM adapts fastest. On Task 2, HER leads on reward while LCM loses its edge under delayed reward. Meanwhile, mean performance gaps sit near 1.0 for every method. That indicates a large settled-state deficit, not a minor tuning gap.
Notably, PPO and HER generally adapt only in the first configuration. They then fail to adapt in later regimes, even without label signals.
Use Cases with Examples
In practice, MORPHEUS suits several reader roles. For AI engineers, it tests whether an agent detects regime shifts without labels. For example, demand switches from low to bursty, and the policy must adapt with no signal.
For data scientists, it stresses delayed credit assignment. For example, On-Time In-Full delivery is observable only days after the dispatch decision. For software engineers, the TypeScript plugin format allows swapping rewards or toggling observability without changing dynamics.
Strengths and Weaknesses
Strengths
- Persistent worlds with no resets, matching deployed enterprise systems.
- Parameterisable, reproducible regime shifts for fair cross-algorithm comparison.
- Rewards from native operational verifiers, needing no external annotation.
- Open-sourced evaluation code.
Weaknesses
- Only two of five environments are evaluated so far.
- The upper bound assumes zero failures, so it stays optimistic.
- Shifts are externally triggered, not driven by compounding decisions.
- Reward weights are research variables, not validated industry objectives.
What it means
Engineers building agents for real-world operations now have a tool to test resilience against changing conditions. The platform forces models to learn continuously rather than relying on static training data. This approach highlights where current algorithms fail when faced with the unpredictable nature of actual enterprise systems.
