Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B

En 30 secondes

Ce qui s'est passé

A controlled experiment shows simple repeated sampling outperforms or matches self-refining methods like Self-Refine and Reflexion when token budgets are equal across 1.5B to 7B models.

Pourquoi ça compte

Matters for engineers choosing inference strategies for math problems and evaluating whether complex reasoning methods justify their computational cost.

Vigilance

Study limited to two math benchmarks with 150 questions each; results may not generalize to other domains, languages, or task types beyond mathematics.