Loading...
Eval-Driven Development (Agent CI)(EDD)
The development-lifecycle discipline of building agents and prompts against evals first. Curated golden datasets (roughly 50 to 500 cases weighted toward known failure modes) are version-controlled alongside the prompts, run as a regression suite on every pull request, and merge or promotion is gated on metric thresholds. The pipeline pins model and judge versions so a silent provider-side model update is detected as a regression instead of being quietly absorbed, and stages its checks as lint, then offline eval, then a cost gate before a change can merge. Distinct from agent-observability-tracing, which is runtime telemetry rather than pre-merge gating, and from fixed public benchmarks, which are not team-owned regression suites in CI.
Loading technique guide…
Eval-Driven Development (Agent CI)(EDD)
The development-lifecycle discipline of building agents and prompts against evals first. Curated golden datasets (roughly 50 to 500 cases weighted toward known failure modes) are version-controlled alongside the prompts, run as a regression suite on every pull request, and merge or promotion is gated on metric thresholds. The pipeline pins model and judge versions so a silent provider-side model update is detected as a regression instead of being quietly absorbed, and stages its checks as lint, then offline eval, then a cost gate before a change can merge. Distinct from agent-observability-tracing, which is runtime telemetry rather than pre-merge gating, and from fixed public benchmarks, which are not team-owned regression suites in CI.
Loading technique guide…