Everything in Moderation: Per-Domain Coverage Optima and Alignment-Resistant Domain Gaps in Multi-Domain Mid-Training

In 30 Sekunden

Was passiert ist

Researchers found that each domain has an optimal data allocation band of 10-40% during mid-training, and alignment cannot fix suboptimal domain composition choices.

Warum es zählt

Engineers designing multi-domain training pipelines need to know that data composition decisions made during mid-training are largely irreversible by later alignment.

Achtung

Study uses logical reasoning tasks on one model size; results may not generalize to other domains, model scales, or real-world training scenarios.