← overview
Inside post-training · the RL loop · source snapshots 17, 21 and 22 September 2026
The loop every production post-training stack implements
Every stack read for this review decomposes into the same roles: a trainer, a rollout service, a weight-sync path, a queue between them, environment and reward services, and an orchestrator. Select a component for the implementations that document it and the audited tasks that touch it.
L1
The loop
Trajectories flow forward, weights flow back; the badge on each box counts the audited tasks that touch it. The view is a synthesis from slime, verl, SkyRL, AReaL, NeMo-RL and MiMo-V2.6 documentation, not one repository's class diagram.
Five components and two railsselect a box or rail · badge = audited tasks touching it