AgileRL Arena v1.0 releases manifest-driven RL training with LoRA/GRPO support
AgileRL Arena v1.0 has been released, introducing a manifest-driven approach to reinforcement learning (RL) training that supports both local and cloud environments. The update centers on a new architecture where algorithms are constructed from specifications via paradigm builders, allowing for flexible configuration of training loops based on flags such as off-policy, offline, or bandit modes.
Key points
- AgileRL Arena v1.0 introduces manifest-driven RL training with support for local and cloud execution environments.
- New paradigm builders allow algorithm construction from specs, integrating LoRA and GRPO for LLM finetuning.
- Dependencies updated to PEFT 0.20.0 and liger-kernel 0.8.2, with specific fixes for Mamba model compatibility.
A key feature of this release is the integration of Large Language Model (LLM) finetuning capabilities, specifically supporting LoRA and GRPO methods. The framework now depends on agilerl-arena=1.0.0 and includes updated dependencies like PEFT 0.20.0 and liger-kernel 0.8.2. Notably, the PEFT update addresses compatibility issues with Mamba mixer architectures by excluding specific modules from LoRA adaptation, ensuring stable training with mixed precision.
The release also improves developer experience through new CLI commands for manifest validation and schema inspection. By isolating training specs within agilerl.arena.models and rejecting unknown keys, the framework aims to provide a more robust and predictable environment for building and deploying RL agents and LLM finetuning pipelines.
AgileRL Arena v1.0: manifest-driven RL training (local or cloud), with LoRA/GRPO LLM finetuning
github.com · 11 September 2026agilerl-arena v1.0.0: build algorithms from specs via paradigm builders
Features
-
build algorithms from specs via paradigm builders spec.build_algorithm() delegates toagilerl.builders . Specs remain arena field subclasses with construction wrappers. Training loops still come from the spec. Builderbuild() takes anAlgorithmBuildRuntime for the population slot, device, HPO, and checkpoint.
dispatch local training through paradigm strategies Training loops are selected by agilerl.strategies.select_strategy from the spec's paradigm flags (off_policy ,offline ,bandit ,env_type ).LocalTrainer uses that layer. Specs still exposeget_training_fn andget_training_kwargs . Multi-agent fitness logs take a per-agent dict.
define training specs only in agilerl-arena The training manifest lives in agilerl.arena.models . The framework imports those classes as the specs; it does not subclass them to addmake_env /init_buffer /build . Builders and strategies sit beside the specs. Unknown keys are rejected. Defaults match the algorithm constructors.evo_steps is optional.LocalTrainer takes networks and HPO as arguments. New CLI:arena manifest validate andarena manifest schema .agilerl now depends onagilerl-arena>=1.0.0,<2.0 .
Fixes
- isolate dummy algorithm specs from the global registry A unit test no longer leaves a dummy spec on the global algorithm registry, which made later tests that walk every registered spec fail depending on collection order. Training strategy types now include LLM and bandit envs and match the fitness values the loops return.
Other
- bump peft to 0.20.0 and liger-kernel to 0.8.2 PEFT 0.20 rejects LoRA on Mamba mixer out_proj and conv1d. adapt_lora_config_for_model excludes those modules. hydra-core stays on 1.3.x. LLMAlgorithm backward stays under AMP so fp16 checkpoint recompute matches LoRA dtypes.
This text was published by github.com . It is reproduced here with attribution so you can read it in full; the rights remain with the publisher. Read it at the source ↗
Coverage and discussion
1 source- Reddit discussion reddit.com
The headline, key points and digest above were generated by Digest AI's editorial model from the linked sources. Automated summaries can contain errors: the sources are the record. Spotted a mistake? Tell us.
Comments
via GitHub Discussions