Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost

In this articleTL;DRTable of ContentsTraining inside agent harnessesGenerating custom RL tasks on Hugging FaceTesting the filters with short training runs Introducing harness…

By Vane November 20, 2026 6 min read
Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost


Introducing harness adapters: adapt your favorite model to your harness to improve performance and cost

Training a Qwen3.8-27B model with a custom RL adapter raised its Terminal-Bench 2.1 score from 39% to 55% while cutting average token usage from 2.9K to 1.5K per task.

Standard models target general performance rather than specific agent environments. Combining a generic model with a specific harness leaves efficiency on the table for the work you actually do. Reinforcement learning allows you to train a custom adapter that extracts maximum value from your model for your specific harness and task combination.

Earlier experiments with data-analysis and code-building showed that RL on your own task distribution inside your own harness yields more accurate and efficient models. This project pushed that further by testing the approach at 27B parameters on harder real-world software engineering tasks across three harnesses: Pi, OpenCode, and Hermes Agent. The team also shared a fully reusable recipe so others can reproduce results on their own combinations.

TL;DR

  • The team trained one LoRA adapter for Qwen3.8-27B per agent harness (pi, Hermes Agent, OpenCode) using RL inside the harness itself.
  • Datasets, training, and agent rollouts all ran on the Hub in Jobs using TRL’s
    AsyncGRPOTrainer

    , OpenEnv, and harbor.

  • Each harness adapter beats the base model under its own harness on both benchmarks for performance and efficiency.
  • The three LoRAs merge into a single super adapter that runs unchanged under pi, Hermes Agent, and OpenCode.
  • All artifacts (adapters, GGUFs, RL tasks, safeagents plugin) are available through this Hugging Face collection.

Table of Contents

  • Training inside agent harnesses
  • Generating custom RL tasks on Hugging Face
  • Testing the filters with short training runs
  • AsyncGRPOTrainer

    for more efficient GRPO training

  • Training setup
  • Super-adapter: merging all adapters into one
  • Final results
  • Open question: What did the model really learn to do?
  • Closing words and future directions

Training inside agent harnesses

The team leveraged the Harbor integration in OpenEnv and TRL’s OpenEnv guide to train the model inside the chosen harnesses. They built ready-to-use docker images for all RL tasks that come with the target harnesses pre-installed. The final images used Pi

1.5.2

, OpenCode

2.2.7

and Hermes Agent

v0.22.6

.

Most RL frameworks train a re-implementation of the agent loop where the agent is written against the framework’s own interface. Here the harness runs exactly as it ships, and the trainer learns from the exact tokens it produced, recorded by a proxy between the harness and vLLM. Recent work moves in the same direction: Agent Lightning v1.0 keeps the deploy-time harness in the loop through a proxy, and SkyRL’s Harbor integration trains on Harbor trials, converting their chat histories into tokens for the trainer.

Generating custom RL tasks on Hugging Face

The recipe created a custom dataset for agentic coding with the Hub, OpenEnv, and Repo2RLEnv. It combined task retrieval from existing RL envs, task generation with Repo2RLEnv, task validation with OpenEnv, and sampling into a single data pipeline. This produced the highest quality set of validated and relevant tasks to improve Qwen3.8-27B on agentic coding.

RL training needs tasks the agent can attempt, a verifier that scores each attempt, and evidence that the verifier is right. A verifier that accepts wrong answers teaches the wrong behavior. A task that is already feasible will not teach the model anything.

The task set came from three sources: two on the Hugging Face Hub and a third generated with Tasksmith, a pipeline in Repo2RLEnv. Every task passed an eligibility screen with

openenv validate

, and the same difficulty check against the base model, Qwen3.8-27B.

Three sources

Each task uses the Harbor format, which Terminal-Bench 2.1 also uses. A broad search of the Hub for Harbor environments returned about 59,000 tasks, but they included duplicates, evaluation sets and tasks marked as not for training. The team filtered down to just two Hub sources built for training and generated a third set themselves:

  • SWE-smith (Python, Go) injects bugs into real Python and Go repositories, with an LLM rewrite or a mutation such as a flipped operator. The tests that catch each bug become the task’s tests. It adds volume.
  • SWE-Gym, packaged as Harbor tasks in TaskTrove, turns real GitHub issues and their merged fixes into tasks. It adds real-world bugs.
  • Tasksmith generates tasks from merged pull requests. An LLM agent writes the instruction and tests, and Tasksmith checks the result: the reference and a valid alternative must pass, while the unfixed code and a known-wrong solution must fail.

The team used Tasksmith to target gaps. They mapped the skills that the target benchmarks test (Terminal-Bench 2.1, the hard subset of SWE-bench Pro, and DeepSWE), then chose pull requests that exercise them, such as HTTP streaming, filesystem operations and serialization. From 30 merged pull requests, Tasksmith produced 24 tasks in 10 Python repositories that passed its checks, 17 of them on a second run. Its checks do not replace the team’s own checks, so these tasks went through the same pipeline as the rest.

Validation with

openenv validate

A metadata screen ran first. It kept tasks from training splits with permissive licenses. It dropped duplicates, tasks marked as not for training, the 253 tasks in the target benchmarks, and any task from a repository those benchmarks use. The team then ran the semantic checks from

openenv validate

on every task in fresh trials:

  • The reference solution earned the maximum reward (
    semantic.oracle_max

    ).

  • Doing nothing scored clearly lower (
    semantic.floor_gap

    ).

  • No canary scored more than doing nothing (
    semantic.canary_floor

    ).

  • A replay of the reference gave the same reward (
    runtime.episode_determinism

    ).

Each trial ran as its own Hugging Face Job, which let the team validate dozens of tasks at once.

openenv validate

offers the same option for any Harbor task:

openenv validate path/to/task --sandbox hf-sandbox

858 of 1,127 Hub candidates passed. Wrong-answer controls caught defects, including a Tasksmith task that accepted an incomplete implementation and a Terminal-Bench 2.1 verifier (regex-log) that accepted known-wrong answers. Most failures were infrastructure problems, such as images that did not start. Failed tasks stayed in a rejected split with the failing check recorded.

Difficulty check with the base model

A valid task can still teach nothing. GRPO (group relative policy optimization) scores each attempt against the other attempts at the same task, so a task where every attempt passes, or every attempt fails, gives no learning signal. Before training, Qwen3.8-27B attempted each task 4 times, or 8 times for a stratified sample of six tasks per repository.

Of 442 scored tasks, Qwen3.8-27B solved 151 every time and 120 never. The 171 tasks it solved only sometimes formed the qualified set. The model solved most Go tasks every time, failed every attempt on more than a third of the SWE-smith Python tasks, and landed in the middle on 73% of the SWE-Gym tasks.

This example showed how the team filtered and validated 1,127 tasks to 278 which they included directly in the final dataset. The pipeline then took the remaining 522 rejected tasks, repaired and complexified them with Tasksmith and repeated validation.

Testing the filters with short training runs

The pipeline applies two filters: validation removes broken tasks, and the difficulty check keeps tasks the base model solves only sometimes. The team tested whether those filters change what the model learns. They trained four short runs of up to 50 steps, each on tasks from a different stage of the pipeline, and compared them with the base model on Terminal-Bench 2.1.

Each run trained Qwen3.8-27B with LoRA and GRPO, using TRL’s

AsyncGRPOTrainer

, on 32 tasks. All four used the same agent harness (OpenCode with up to 48 model calls), the same settings and one 8×H200 Hugging Face Job, and the reward was the verifier’s pass or fail. The task sets differed in how they were filtered:

  • Unvalidated: 24 accepted tasks plus 8 that failed validation.
  • Validated: 32 accepted tasks, chosen without regard to difficulty.
  • Unqualified: 32 accepted tasks that the base model always or never solves.
  • Qualified: 32 accepted tasks that the base model solves on some attempts.

The dataset

harness-hub/agentic-coding-envs

holds 871 tasks from 35 repositories. Hub tasks are capped at 40 per repository, so no single project dominates. The dataset has two views: every accepted task, and the qualified subset for GRPO training. Rejected attempts stay in a separate config.

SourceTask typeTasksQualified
SWE-smithInjected bugs in Python and Go717144
SWE-Gym (TaskTrove)Real GitHub issues13027
TasksmithGenerated from merged pull requests2422
Total871171

Scroll to Top