Less Loss, Fixed Weights: Logit Corrections for Forgetting

In 30 Sekunden

Was passiert ist

Researchers tested whether fixed token-level adjustments can recover performance lost when language models are fine-tuned or retrained on new domains.

Warum es zählt

Matters for teams deploying instruction-tuned or domain-adapted models that need to maintain baseline performance on general text.

Achtung

Simple fixes recovered at most 36% of losses; models retrained on math recovered almost nothing, suggesting the approach has fundamental limits.