The best mid-size model to fine-tune was the third-best model to prompt

За 30 секунд

Что произошло

Nemotron 3.5 Lightning ranked third in zero-shot performance but first after fine-tuning across six tasks, beating a 550B teacher model on four tasks.

Почему это важно

Engineers choosing which mid-size model to fine-tune for production features should prioritize fine-tuning capability over base model zero-shot rankings.

На что обратить внимание

Results are tight with many ties; text2sql performance at 0.850 is weak; small test sets mean sampling noise significantly affects rankings and conclusions.