Agent Lightning
Trains the model inside an agent you have already built, by sitting between it and the model instead of rewriting it
What is Agent Lightning?
The usual objection to reinforcement learning for agents is that it means rebuilding the agent as a training environment, at which point you are no longer improving the thing you run. Agent Lightning avoids that by putting a proxy where the model endpoint used to be: the agent keeps its own tools, prompts, control flow and environment, and every interaction that passes through the proxy becomes training data. A trainer updates the policy, a controller runs the agent locally or as Kubernetes jobs. Microsoft reports taking a 9B model from 41.8% to 56.4% on SWE-bench Verified using 6,000 samples, and publishes that pipeline. This is GPU work with a training stack behind it, not a library you add to an application.
What can you do with Agent Lightning?
- Train the agent you already run — The agent needs no changes: its tools, prompts and control flow stay in the loop, so what improves is the system you actually deploy.
- Collect training data from ordinary traffic — A proxy in front of the model turns each interaction into a training sample, so the data comes from real runs rather than a synthetic environment.
- Read the whole framework if you need to — The maintainers put the code at roughly 3,500 lines, which is small enough to follow when a training run behaves oddly.
- Run rollouts as Kubernetes jobs — Agents can be launched directly as jobs on a cluster, so collecting experience scales without a separate sandbox service in between.
- Follow a complete published recipe — The coding-agent example ships the whole pipeline — data cleaning, reward-hacking prevention and training scripts — rather than a fragment to reconstruct.
Before you choose Agent Lightning
- This is model training. It expects GPUs and a reinforcement-learning stack installed against a matching CUDA build — a very different commitment from adding a library to an application.
- Version 1.0 was a complete rewrite and earlier releases live on a separate branch, so material written before it describes a different system.
Frequently asked questions
Is Agent Lightning free for commercial use?
Agent Lightning is released under the MIT licence — OSI-approved open source, which permits commercial use.
How can Agent Lightning be deployed?
Agent Lightning is available as Self-hosted.
Documentation
Reproduced from the microsoft/agent-lightning README, published under MIT. Read the original ↗
Agent Lightning was completely refactored in v1.0. For legacy releases earlier than v1.0, see this branch.
⚡ Key Features
- 🪶 ~3,500 lines of code: We treat simplicity as the first principle.
- 🧩 Train with real agent harnesses: Agents interact with the model through the Agent Lightning v1.0 proxy with ZERO changes, while keeping tools, context, control flow, and environments in the loop.
- ☸️ Native Kubernetes support: Run agents directly as Kubernetes Jobs without relying on external sandbox services.
- 💻 Full coding agent training example: Using only 6K training samples, an end-to-end Qwen3.5-9B workflow improves SWE-bench Verified from 41.8% to 56.4%, a gain of 14.6 percentage points. We release the full pipeline, including data cleaning, reward-hacking prevention, and training scripts.
⚡ Installation
The following is an example installation on a CUDA 13.0 machine:
cd <this-repo>
uv sync
bash scripts/setup_verl.sh 0.8.0 cu130See the Installation Guide for details.
⚡ Architecture
Agent Lightning v1.0 keeps the training architecture simple with three lightweight components:
- Trainer: Runs
verland vLLM, builds training samples, and updates the policy. - API Gateway: Proxies model requests and captures training data.
- Rollout Controller: Runs agents locally or as Kubernetes Jobs.
The Trainer creates rollouts, the Controller launches agents, and the Gateway turns interactions into training data, while agents continue to run with their real harnesses.
⚡ Results
We evaluate Agent Lightning v1.0 across several practical training domains, including Search R1, LLM-in-Sandbox, and Coding Agent. Pure RL delivers substantial improvements across all three domains, as shown below.
⚡ Documentation
| Section | Content |
|---|---|
| Installation | Base environment and verl GPU stack |
| Quick Start | Local first run and end-to-end flow |
| Basics | Components, rollouts, events, and trajectories |
| Trainer Configuration | verl integration and trace aggregation |
| API Gateway Configuration | Gateway and model proxy settings |
| Controller Configuration | Local and Kubernetes runners |
| Asynchronous Training | Collocated async collection and pause/drain |
⚡ Examples
| Example | Description |
|---|---|
| Calc-X | POC math reasoning example with AutoGen and MCP calculator tools, requiring only one GPU. |
| GSM8K | POC grade-school math reasoning example. |
| ScienceWorld | Interactive science tasks in a text-based environment. |
| Search-R1 | Multi-turn retrieval and reasoning agent. |
| LLM-in-Sandbox | General agent with computer and code execution tools. |
| Coding Agent | Coding agent trained with repository tests. |
⚡ Articles
- 8/19/2026 Agent Lightning v1.0: Towards Harnessed Agentic RL technical report.
- 12/17/2025 Adopting the Trajectory Level Aggregation for Faster Training Agent-lightning blog.
- 11/4/2025 Tuning ANY AI agent with Tinker ✕ Agent-lightning Medium. See also Part 2.
- 10/22/2025 No More Retokenization Drift: Returning Token IDs via the OpenAI Compatible API Matters in Agent RL vLLM blog. See also Zhihu writeup.
- 8/11/2025 Training AI Agents to Write and Self-correct SQL with Reinforcement Learning Medium.
- 8/5/2025 Agent Lightning: Train ANY AI Agents with Reinforcement Learning arXiv paper.
- 7/26/2025 We discovered an approach to train any AI agent with RL, with (almost) zero code changes. Reddit.
- 6/6/2025 Agent Lightning - Microsoft Research Project page.
⚡ Community Projects
- DeepWerewolf — A case study of agent RL training for the Chinese Werewolf game built with AgentScope and Agent Lightning.
- AgentFlow — A modular multi-agent framework that combines planner, executor, verifier, and generator agents with the Flow-GRPO algorithm to tackle long-horizon, sparse-reward tasks.
- Youtu-Agent — Youtu-Agent lets you build and train your agent with ease. Built with a modified branch of Agent Lightning, Youtu-Agent has verified up to 128 GPUs RL training on maths/code and search capabilities with steady convergence. Also check the recipe and their blog Stop Wrestling with Your Agent RL: How Youtu-Agent Achieved Stable, 128-GPU Scaling Without Breaking a Sweat.
⚡ Citation
If you use Agent Lightning v1.0 in your research or projects, please cite the technical report:
@misc{he2026agentlightningv10harnessed,
title={Agent Lightning v1.0: Towards Harnessed Agentic RL},
author={Zhiyuan He and Siwei Zhang and Zhiwen Zhou and Yuqing Yang and Yu Kang and Yuge Zhang and Luna K. Qiu and Tin Yan Tsui and Jiahang Xu and Chong Luo},
year={2026},
eprint={2608.17528},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2608.17528},
}For the original Agent Lightning paper, please use:
@misc{luo2025agentlightningtrainai,
title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning},
author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang},
year={2025},
eprint={2508.03680},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2508.03680},
}⚡ Contributing
This project welcomes contributions and suggestions. Start by reading the Contributing Guide for recommended contribution points, environment setup, branching conventions, and pull request expectations. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
⚡ Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft’s Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party’s policies.
⚡ Responsible AI
This project has been evaluated and certified to comply with the Microsoft Responsible AI Standard. The team will continue to monitor and maintain the repository, addressing any severe issues, including potential harms, if they arise.
⚡ License
Agent Lightning v1.0 is released under the MIT License.
Other options in Prompt & Optimization
FaissVector & StorageIt is built around an index that holds a set of vectors and searches them, and the dozens of available index structures exist because the trade-offs are real: search time against result quality against memory per vector against how long the index takes to build, and whether it needs training data at all. Some methods keep only a compressed representation and never the original vectors, which is how a single server reaches billions of them. There is no server, no filtering and no access control here — those are what the databases built on top of it add.
DSPyOptimizationYou declare what each step should do and supply a metric; the optimiser searches for the prompts and few-shot examples that maximise it. The payoff is that improving a pipeline becomes measurable rather than superstitious. It requires an evaluation set — without one there is nothing to optimise against.
OpikObservabilityA decorator on a function or a wrapper around a client is enough to start recording traces, and from there the same data feeds experiments that score responses automatically instead of by hand — hallucination, answer relevance, context recall and the rest. More than thirty integrations cover the usual frameworks and providers, and the whole platform can run locally or on Kubernetes rather than only as a hosted service. Its difficulty is not capability but overlap: it competes directly with two platforms already in this catalogue, so the choice turns on integrations and hosting rather than on what is missing.