PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

In 30 Sekunden

Was passiert ist

PPL-Factory selects training data for fine-tuning language models using task-aware perplexity scores and budget constraints, achieving full-data accuracy with only 10% of samples.

Warum es zählt

Engineers fine-tuning large language models on reasoning tasks like math problems need to reduce computational costs without sacrificing downstream performance.

Achtung

Results demonstrated on GSM8K and MATH datasets; effectiveness on other task domains and model sizes remains unclear from this paper.