The best mid-size model to fine-tune was the third-best model to prompt

In 30 Sekunden

Was passiert ist

Nemotron 3.5 Lightning ranked third in zero-shot performance but first after fine-tuning across six tasks, beating a 550B teacher model on four tasks.

Warum es zählt

Engineers choosing which mid-size model to fine-tune for production features should prioritize fine-tuning capability over base model zero-shot rankings.

Achtung

Results are tight with many ties; text2sql performance at 0.850 is weak; small test sets mean sampling noise significantly affects rankings and conclusions.