Generated by Codex with GPT 5.6 Sol High

An agent is not just a language model. Its behavior also depends on the harness that manages context, tools, retries, subagents, and execution environments. That creates an awkward gap for reinforcement learning: many training systems own the interaction loop themselves, so engineers must reproduce the production harness inside the trainer. In “Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses,” published by the official Microsoft Research Blog, Microsoft Research Asia proposes a cleaner boundary. The deployed harness stays intact and participates directly in training.

Train the system that will actually run

Traditional agentic RL treats a rollout as one continuous sequence: the model produces an action, the environment returns an observation, and the trainer appends both to the context. That assumption breaks down for mature agents. A coding harness may summarize context, launch subagents, make several model calls through different paths, or apply its own tool protocol. Reimplementing those behaviors inside a training framework is both expensive and risky because the training-time agent can drift from the deployed one.

Agent Lightning instead inserts an OpenAI-compatible proxy between the harness and the model. Existing agents keep their orchestration code and point their model endpoint at the proxy. The proxy associates each request and response with a rollout while recording the prompts, sampled tokens, log probabilities, and events needed for learning. Microsoft calls this pattern Harnessed Agentic RL: the harness that runs in production is also the harness that generates training experience.

This boundary makes integration much smaller, but it also exposes statistical problems hidden by the usual single-trajectory abstraction. One rollout can become a variable number of samples because of retokenization, context compaction, or subagent calls. If advantages or losses are averaged per sample, harnesses that generate more calls receive extra weight for reasons unrelated to task quality. Agent Lightning therefore preserves rollout-level accounting when computing advantages and normalizing loss. That is a subtle but important lesson: once orchestration changes the shape of the data, the training objective must still reflect the unit of behavior being rewarded.

A small control plane around existing agents

The roughly 3,500-line framework has three main components. An API gateway stores rollout state and model events while serving as the LLM proxy. A rollout controller starts agents as local processes or Kubernetes jobs and tracks their lifecycle independently from training. A customized trainer, built on verl, requests rollouts, waits for results, and uses a sample adapter to assemble training inputs.

The separation keeps agents outside the trainer’s process and dependency graph. It also makes the system practical for real harnesses such as mini-SWE-agent or OpenHands: the harness remains an ordinary workload, while the control plane observes its calls and converts them into learning data. Native Kubernetes execution lets teams reuse their own clusters instead of depending on a commercial sandbox fleet, which improves reproducibility and gives operators direct control over cost and isolation.

Agent workloads create another systems problem: rollout duration varies widely. Synchronous RL leaves GPUs idle while a batch waits for its slowest agent, but fully asynchronous designs often reserve separate GPU pools for inference and updates. Agent Lightning’s Collocated Async RL shares one pool. The gateway accepts rollouts until enough experience is ready, pauses new requests, drains in-flight calls, performs the update, and then resumes serving. The external harness does not need to know that the model has switched between inference and training phases. Microsoft reports about a twofold end-to-end speedup over synchronous RL while using fewer GPUs than a conventional asynchronous setup.

What the result shows—and does not show

The team tested the design with a pipeline built from SWE-smith, mini-SWE-agent, and Qwen3.5-9B. Using about 6,000 training samples, RL increased Pass@1 on SWE-bench Verified from 41.8% to 56.4%. Experiments also supported the rollout-level treatment: combining rollout-level advantages with rollout-level loss normalization produced stronger validation reward and more stable policy entropy than sample-level accounting.

The headline score is useful evidence, but the more durable contribution is architectural. Agent Lightning turns an agent harness into an external client of a narrow training control plane instead of forcing the trainer to absorb the agent’s full implementation. That keeps training closer to deployment, lets different harnesses share infrastructure, and makes changes to tools or context management immediately visible to learning.

The broader engineering takeaway is to preserve the real system boundary when optimizing a compound AI application. If a model’s behavior depends on orchestration, training a simplified imitation can optimize the wrong thing. Instrument the production-shaped loop, keep rollout identity intact across many calls, and design scheduling around the irregular workload agents actually create.