Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B

In 30 Sekunden

Was passiert ist

A controlled experiment shows simple repeated sampling outperforms or matches self-refining methods like Self-Refine and Reflexion when token budgets are equal across 1.5B to 7B models.

Warum es zählt

Matters for engineers choosing inference strategies for math problems and evaluating whether complex reasoning methods justify their computational cost.

Achtung

Study limited to two math benchmarks with 150 questions each; results may not generalize to other domains, languages, or task types beyond mathematics.