Spatial downsampling through aggregation
Pooling layers reduce spatial dimensions by aggregating regions of the feature map into single values. Max pooling takes the largest value in each non-overlapping region (typically 2x2), discarding the others. Average pooling takes the mean. A 64x64 image pooled with 2x2 becomes 32x32. Pooling reduces computation, memory, and parameters for downstream layers. It also provides translation invariance: if an object shifts by a pixel, max pooling in that region still captures its presence. Early CNNs heavily used pooling; modern architectures use it less and prefer strided convolutions for downsampling, which are learnable and often more expressive.
Tradeoffs with learned alternatives
Max pooling is nonsmooth and non-differentiable (gradients are sparse), but the flow-through version used in backprop approximates the gradient. Average pooling is smooth but loses spatial specificity: the presence of a high-activation edge is collapsed to the same pool value as a high-activation region elsewhere. Strided convolution (advancing the kernel by 2 steps instead of 1) also downsamples and is learnable, but adds parameters. Modern best practice for CNNs uses strided conv for spatial reduction with occasional max pooling in specific layers. Vision transformers sidestep the question entirely by patchifying the image upfront, avoiding explicit pooling until the end (global average pooling before classification).