Encoder–decoder Transformers

训练方式

Teacher-forced target-token likelihood is common. Text-to-text denoising or span corruption lets many tasks share the same input/output interface.

推理运行方式

The source is encoded once; the decoder generates the target autoregressively while attending to stable source states. Beam search is useful in some bounded tasks but is not universally best.

优势

• •Clear separation between source understanding and target generation
• •Strong fit for translation, summarization, transcription, and transformation
• •Source memory is reused throughout decoding

权衡

• •Two stacks can increase parameters and deployment complexity
• •Target decoding remains serial
• •May be less convenient than one decoder-only interface for heterogeneous chat tasks

适用场景

• Output is tightly conditioned on a distinct input
• Source and target have different structures or modalities
• Faithful transformation matters more than open-ended continuation

应避免或质疑的场景

• A single general chat interface is the overriding requirement
• No distinct source sequence exists
• A non-generative encoder head is sufficient