基于带标签的输入-输出示例调整智能体组件，并配以显式验证和回退行为
Loading technique guide…
模式： 通过在目标领域标注数据上的有监督训练,将预训练模型适配到特定任务
原因： 在利用模型已有知识的同时针对目标任务专精,以合理的训练成本取得高性能
关键洞察： 预训练模型的迁移学习结合任务专属监督信号,可在泛化与专精之间取得最佳平衡
通过这些精选资源加深理解
Fine-Tuning Pre-trained Language Models: Weight Initializations, Data Orders, and Early Stopping (Dodge et al., 2020)
How to Fine-Tune BERT for Text Classification? (Sun et al., 2019)
Universal Language Model Fine-tuning for Text Classification (Howard & Ruder, 2018)
Attention Is All You Need (Vaswani et al., 2017)
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Devlin et al., 2018)
Language Models are Few-Shot Learners (Brown et al., 2020)
T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (Raffel et al., 2019)
RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019)
LoRA: Low-Rank Adaptation of Large Language Models (Hu et al., 2021)
Prefix-Tuning: Optimizing Continuous Prompts for Generation (Li & Liang, 2021)
The Power of Scale for Parameter-Efficient Prompt Tuning (Lester et al., 2021)
AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning (Zhang et al., 2023)
Domain-Adversarial Training of Neural Networks (Ganin et al., 2016)
Deep Domain Confusion: Maximizing for Domain Invariance (Tzeng et al., 2014)
Unsupervised Domain Adaptation by Backpropagation (Ganin & Lempitsky, 2015)
AdaBound: Adaptive Gradient Methods with Bound for Domain Adaptation (Luo et al., 2019)
QLoRA: Efficient Finetuning of Quantized LLMs (Dettmers et al., 2023)
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention (Zhang et al., 2023)
Instruction Tuning for Large Language Models: A Survey (Zhang et al., 2023)
DoRA: Weight-Decomposed Low-Rank Adaptation (Liu et al., 2024)
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (Wang et al., 2018)
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems (Wang et al., 2019)
Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models (Srivastava et al., 2022)
HELM: Holistic Evaluation of Language Models (Liang et al., 2022)
BioBERT: A Pre-trained Biomedical Language Representation Model for Biomedical Text Mining (Lee et al., 2020)
FinBERT: Financial Sentiment Analysis (Araci, 2019)
LEGAL-BERT: The Muppets Straight out of Law School (Chalkidis et al., 2020)
CodeBERT: A Pre-Trained Model for Programming Languages (Feng et al., 2020)