In the news
Async GRPO with LoRA across HF Jobs: a bucket, a proxy, and no NCCL
Hugging Face · Published · 3 min read
In 30 seconds
- What happened
- TRL v1.14 now supports LoRA-only training with AsyncGRPOTrainer, syncing only adapter weights instead of full models across distributed jobs.
- Why it matters
- Engineers scaling reinforcement learning training across separate machines without shared filesystems or NCCL, using Hugging Face Jobs infrastructure.
- Watch out
- Requires careful adapter slot management based on max_staleness to prevent silent policy mismatches; vLLM version pinning and specific runtime flags are essential for correctness.
- lora
- grpo
The patterns behind this
- Reinforcement Learning from Human Feedback
- Machine Learning Model-Based Routing
- Reinforcement Learning from AI Feedback
Each one covers how the technique works, when it earns its cost, and where it breaks.
The Agent Architect
One pattern, one tradeoff, one production failure story. A short weekly briefing for people building agentic systems.
Weekly email, one-click unsubscribe. We only use your address to send the briefing.