PPL-Factory: Task-Aware and Budget-Aware Data Selection from Language Modeling to Reasoning

За 30 секунд

Что произошло

PPL-Factory selects training data for fine-tuning language models using task-aware perplexity scores and budget constraints, achieving full-data accuracy with only 10% of samples.

Почему это важно

Engineers fine-tuning large language models on reasoning tasks like math problems need to reduce computational costs without sacrificing downstream performance.

На что обратить внимание

Results demonstrated on GSM8K and MATH datasets; effectiveness on other task domains and model sizes remains unclear from this paper.