Zero Mismatch, Simple Async RL: REINFORCE Zero, DeepSeek-V4.1 and Kimi K3.1

30秒で要点

何が起きたか

REINFORCE Zero enables stable asynchronous reinforcement learning by aligning computational kernels between rollout and training, then applying unclipped importance sampling to handle weight lag.

なぜ重要か

Engineers optimizing large language model training care when they need faster asynchronous RL that maintains training stability without sample rejection or gradient clipping.

注意点

Kernel alignment adds 6.8% computational overhead compared to standard kernels, and requires model-specific engineering work for each new architecture or hardware platform.