現在のコンテキスト内の指示や実例に基づいてモデルを条件付けする手法で、モデルのパラメータは更新しない
Loading technique guide…
パターン: パラメータ更新なしにモデルが学習できるよう、プロンプトにタスク例を提示する
理由: タスクへの迅速な適応を可能にし、学習不要でモデルのパターン認識を活かして即座に性能を発揮する
重要なポイント: モデルは文脈内のデモンストレーションから学習し、推論時に暗黙の勾配降下を行う
厳選された資料で理解を深めましょう
Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent (Dai et al., 2022)
Transformers Learn In-Context by Gradient Descent (von Oswald et al., 2022)
In-context Learning and Induction Heads (Olsson et al., 2022)
What learning algorithm is in-context learning? Investigations with linear models (Akyürek et al., 2022)
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Wei et al., 2022)
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models (Zhou et al., 2022)
Self-Consistency Improves Chain of Thought Reasoning in Language Models (Wang et al., 2022)
Large Language Models are Zero-Shot Reasoners (Kojima et al., 2022)
Measuring and Narrowing the Compositionality Gap in Language Models (Press et al., 2022)
Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? (Min et al., 2022)
Ground-Truth Labels Matter: A Deeper Look into Input-Label Demonstrations (Yoo et al., 2022)
Fantastically Ordered Prompts and Where to Find Them (Lu et al., 2021)