Loading...
Progressive Rollout & Shadow Mode(PRS)
The deployment side of shipping an agent or prompt version safely. The candidate first runs in shadow mode, executing on real production traffic while its output is withheld from users and compared offline against the served version, then graduates to a canary on 1 to 5 percent of live traffic scored by online evals against a control, then a progressive ramp, with automatic rollback the moment a monitored metric regresses. Pairing online evals (scoring live traffic in flight) with offline evals catches the long-tail failures a static golden set cannot predict. Distinct from eval-driven-agent-development, which gates changes offline before merge, whereas this governs the online ramp after merge.
Loading technique guide…
Progressive Rollout & Shadow Mode(PRS)
The deployment side of shipping an agent or prompt version safely. The candidate first runs in shadow mode, executing on real production traffic while its output is withheld from users and compared offline against the served version, then graduates to a canary on 1 to 5 percent of live traffic scored by online evals against a control, then a progressive ramp, with automatic rollback the moment a monitored metric regresses. Pairing online evals (scoring live traffic in flight) with offline evals catches the long-tail failures a static golden set cannot predict. Distinct from eval-driven-agent-development, which gates changes offline before merge, whereas this governs the online ramp after merge.
Loading technique guide…