タスクラベルのないデータから表現、クラスタ、または異常スコアを学習し、その後ドメインでの検証を行う手法
Loading technique guide…
パターン: 自己教師あり学習の目的関数を用い、ラベルなしデータで新しいドメインにモデルを適応させる
理由: 豊富なラベルなしデータを活かし、高コストな注釈なしでドメイン固有のパターンを学習する
重要なポイント: 自己教師ありのプリテキストタスクがデータ構造から教師信号を生み出し、効果的なドメイン適応を可能にする
厳選された資料で理解を深めましょう
Self-Supervised Learning: Generative or Contrastive (Liu et al., 2021)
A Simple Framework for Contrastive Learning of Visual Representations (Chen et al., 2020)
Momentum Contrast for Unsupervised Visual Representation Learning (He et al., 2020)
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning (Grill et al., 2020)
BERT: Pre-training of Deep Bidirectional Transformers (Devlin et al., 2018)
RoBERTa: A Robustly Optimized BERT Pretraining Approach (Liu et al., 2019)
DeBERTa: Decoding-enhanced BERT with Disentangled Attention (He et al., 2020)
ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators (Clark et al., 2020)
SimCLR: A Simple Framework for Contrastive Learning (Chen et al., 2020)
SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments (Caron et al., 2020)
SimCSE: Simple Contrastive Learning of Sentence Embeddings (Gao et al., 2021)
SupCon: Supervised Contrastive Learning (Khosla et al., 2020)
BERT: Pre-training of Deep Bidirectional Transformers (Devlin et al., 2018)
SpanBERT: Improving Pre-training by Representing and Predicting Spans (Joshi et al., 2020)
ERNIE: Enhanced Language Representation with Informative Entities (Zhang et al., 2019)
ALBERT: A Lite BERT for Self-supervised Learning (Lan et al., 2019)
CLIP: Learning Transferable Visual Representations from Natural Language Supervision (Radford et al., 2021)
ALIGN: Scaling Up Visual and Vision-Language Representation Learning (Jia et al., 2021)
Florence: A New Foundation Model for Computer Vision (Yuan et al., 2021)
DALLE-2: Hierarchical Text-Conditional Image Generation (Ramesh et al., 2022)