The repeating unit of deep transformers
A transformer block (or transformer layer) is a reusable module that stacks multi-head attention, normalization, and a feedforward network (MLP). The block receives token embeddings, applies attention to mix information across all tokens, applies residual connections to preserve identity, normalizes, passes through a two-layer MLP with an activation function, and outputs refined embeddings. This same block repeats dozens or hundreds of times, creating the depth that gives transformers their power.
Residual connections and normalization enable deep stacking
Modern transformer blocks use residual connections (skip connections) after both the attention and MLP sublayers, allowing gradients to flow directly through layers during backpropagation. Layer normalization before each sublayer (or after, in some architectures) stabilizes training and prevents activation distributions from drifting. These two techniques work together to make it feasible to train models 50+ layers deep without gradient vanishing or exploding, which would be impossible without them.
The MLP component typically expands the hidden dimension by a factor of 4 during the first sublayer, then projects back down. This expansion-projection pattern increases model capacity within each block while maintaining manageable overall complexity.