Two ways to split (or not split) the transformer
The original Transformer (2017) had an encoder and decoder. The encoder processes the input bidirectionally (no causal mask); the decoder attends to both the input (via cross-attention) and generates autoregressively. T5 follows this pattern. GPT-style models use only a decoder, attending to the input as part of the same sequence it generates from.
The encoder-decoder split is natural for tasks like translation, where input and output languages are distinct. Decoder-only is simpler: everything is one left-to-right sequence.
Capacity allocation and cross-attention overhead
In encoder-decoder, roughly half the parameters are in the encoder (understanding input) and half in the decoder (generating output). This specialization can be efficient for tasks with clear input-output separation. Cross-attention (decoder attending to encoder outputs) adds compute but allows the decoder to focus on generation, not input processing.
Decoder-only allocates all parameters to a single sequence understanding task. Modern large models (GPT-4, Llama) are decoder-only, suggesting that unified processing of input and output in a single stack is more efficient at scale.
Task fit and practical differences
Encoder-decoder is better for abstractive summarization, machine translation, and question-answering where the input is read once and the output is generated. Decoder-only is better for in-context learning and long conversations because the same sequence understanding mechanism handles both past outputs and future generation.
Most new models are decoder-only because in-context learning (the ability to adapt to new tasks from examples in the prompt) has proven more valuable than architectural specialization.