Researchers from Google Cloud AI Research, Washington University in St. Louis and UNC Chapel Hill have released EnvHarness. This tool wraps existing static agent benchmarks in a programmable layer that adapts to the agent training on them. LLM agents are shifting from learning static text to learning from interactive environments. Current environments are hand-built and frozen. They behave identically regardless of which agent is acting or how much it has improved. The standard fix is to generate new environments. This approach pins developers to domain-specific pipelines and LLM-written verifiers that require heavy generation and filtering. EnvHarness inverts this process. It wraps an existing environment in plug-in components that operate strictly through the standard reset() and step() interface. These components change where an episode starts, what the agent may do, and what it sees. The underlying simulator, tasks, and human-built verifier remain untouched. An LLM designer called EnvRigger writes these wrappers automatically against flaws it diagnoses in the policy’s own rollouts. Across five benchmarks in four domains, skills mined this way gain up to 9.0 points on held-out tasks with 9.8% fewer execution steps.
Is it deployable
Yes, if you already run an agent eval loop. EnvHarness ships as Apache-2.0 Python with reproduction drivers for six environments. A new benchmark joins by implementing one interface: reset, step, observe, evaluate, get_env_state, save_state, and from_state. Nothing downstream changes. The hard prerequisite is a resettable environment. This rules out live user accounts and physical robots.
Environments that stop teaching
LLM agents now learn less from curated text and more from interactive environments. Those environments are hand-built and static. They behave identically no matter which agent acts or how much it has improved. They cannot target a policy’s weakness and have nothing left to teach once solved.
The usual answer is generating more environments. The EnvHarness paper names two costs. Generation pipelines are domain-specific and do not transfer. LLM-written verifiers must be over-generated and heavily filtered without ever being fully trustworthy.
Wrapping, not authoring
The research team proposes the opposite move. An agent harness makes a frozen LLM capable through plug-in tools, memory and skills. EnvHarness applies that idea to the other side of the loop. It wraps a frozen environment in plug-in components that operate strictly through the standard reset() and step() interface.
Formally, a component is a transformation E’ = w(E) that rewrites the state, action, observation and transition terms. The reward term is deliberately left out. Because no intervention reaches the simulator backend, every reshaped task keeps its original, human-built verifier. Because nothing touches benchmark-specific code, one implementation covers every domain.
Three components ship, and they compose freely:
- Stage replays a fixed action list after reset(), so the episode starts somewhere else. Hiding the target mug in a closed drawer forces search instead of reach.
- Contract installs per-step hooks on the action, transition and observation axes. It blocks an action, rewrites a response, or truncates an observation.
- Chain composes a second environment into the same episode under a shared step budget. The composite verdict is the conjunction of both verifiers.
EnvRigger: the designer loop
Components are policy-agnostic. Choosing them is not. EnvRigger treats the policy as a black box and runs four stages. It observes five baseline rollouts, diagnoses a systemic flaw, writes components as real Python, and validates on five fresh rollouts. Unsolvable and trivially solvable candidates are both rejected. There can be up to five revision rounds per task. Generated hooks compile in an isolated subprocess. A bad mutation becomes a recorded trace rather than a dead run.
Performance
Across ALFWorld, WebArena, SWE-bench Verified, OfficeQA and SpreadsheetBench, skills mined with ReasoningBank-style induction beat both controls on untouched held-out tasks.
ALFWorld average rises from 62.4 to 68.3 against original-environment skills, with +9.0 points on the out-of-distribution split. SWE-bench Verified resolved rate moves 49.88 to 52.58 while average steps fall 55.01 to 49.61. This is the paper’s 9.8% efficiency claim. On SpreadsheetBench and WebArena, skills from unmodified environments land below the no-skill baseline. Reshaping is what makes mining worthwhile. Against domain-specific generators, EnvHarness beats SWE-smith by 2.46 points with 5.11 fewer steps.
Under GRPO on Qwen3-8B-base, RL in reshaped environments beats RL in the originals on three of four metrics. ALFWorld in-distribution moves from 81.4 to 87.9, with a small regression on the OOD split (89.6 to 88.8). Environment scaling reaches 54.79 at 300 environments versus 52.13 for originals and 50.37 for generated ones. This is because the designer co-evolves each batch against the current policy. And asked to steer per-task success rate into [0.4, 0.6], in-band coverage rises from 6% to 80%.
What it means
Developers stop wrestling with brittle, hand-coded simulators. EnvHarness allows them to keep their existing human-built verifiers while using an LLM to dynamically alter the environment’s rules. This means training data remains relevant as agents improve, without needing to rewrite the underlying code for every new task.
