AI agents are beginning to train inside the systems they actually use

Microsoft Research’s Agent Lightning v1.0 release puts a neglected part of an AI agent into the training loop: the harness that manages tools, context and control flow when the agent is actually deployed. Instead of recreating that behaviour inside a reinforcement-learning framework, the live harness continues to operate and sends its model calls through a training proxy.[1]

That change matters because an agent is not just a language model with a longer prompt. The same model can behave very differently when one harness truncates history, another retries failed tools, and a third asks for confirmation before a write. Training a simplified copy and deploying the richer system can produce an avoidable mismatch: the policy learns from one operating loop and is judged inside another.[2]

The harness is part of the policy, even when its weights do not change

In traditional agentic reinforcement learning, the trainer often owns the action-observation loop. It receives an action, advances the environment, appends the observation and asks for the next action. Modern coding and research agents increasingly outsource those decisions to a harness that may compress context, select tools, run subprocesses, coordinate sub-agents or recover from errors. Agent Lightning treats that external machinery as part of the behaviour being trained rather than an implementation detail to be approximated.[2][1]

The proxy records the model requests and responses generated during a real rollout. That creates difficult accounting problems: requests may use different tokenisation, several calls may belong to one outcome, and rewards must be assigned across a trajectory whose timing and control flow belong to the harness. The technical report identifies retokenisation, sample merging, advantage calculation, loss normalisation and backend scheduling as material training choices rather than plumbing.[2]

The reported gain is large, but its denominator matters

Moving from 41.8% to 56.4% is a 14.6-point improvement. Relative to the starting score, that is about 34.9%, calculated as 14.6 divided by 41.8. The relative figure is useful for understanding the size of the change, but it does not mean the agent became 34.9% better at software engineering in general. SWE-bench Verified asks agents to resolve selected repository issues under a defined harness and verifier; deployment work also includes ambiguous requirements, missing tests, security constraints and human review.[1][2]

The release also reports roughly a two-times end-to-end speed-up for its collocated asynchronous reinforcement-learning schedule compared with synchronous training in the described experiments. That is a systems result: it reduces idle time by sharing GPU resources between rollouts and updates. It should not be confused with a two-times improvement in the agent’s coding quality.[1]

EnvHarness changes the other side of the loop

A separate project, EnvHarness, starts from the opposite problem. A benchmark environment may remain static even after an agent has learned its easiest patterns. EnvHarness wraps the environment through standard reset and step interfaces, changing states, rules or observations while retaining the original verifier. Its companion system, EnvRigger, watches trajectories, diagnoses recurring weaknesses, proposes components and validates them with fresh rollouts.[4][5]

Across five benchmarks in four domains, the paper reports gains of up to 9.0 points on held-out instances and 9.8% fewer execution steps. On the project’s SWE-bench example, skills learned in reshaped environments averaged 49.6 steps versus 55.0 for skills learned in the original environments. The shorter trajectory is not automatically better: an agent that stops early because it gives up would also use fewer steps. The relevant evidence is the combination of held-out task success, retained verifiers and reduced steps.[4][5]

What a serious agent team should measure next

The practical opportunity is to maintain three ledgers rather than one leaderboard: model changes, harness changes and environment changes. For each release, a team should record accepted-task rate, cost, elapsed time, tool errors, retries, human interventions and irreversible failures. When performance improves, the ledger should reveal whether the policy learned a better action, the harness prevented a bad one, or the environment supplied a clearer signal.

Agent Lightning and EnvHarness do not show that reinforcement learning will make every production agent reliable. They show a more credible place to run the experiment. The agent can train with the machinery it will actually use, while the environment can evolve without discarding its trusted verifier. That turns agent development from choosing a model and hoping for transfer into an engineering loop whose moving parts can be tested separately.

Report an error or suggest a correction