Underfitting: when the model is too simple
Bias measures how far a model's average prediction is from the true value. A linear regression fit to a nonlinear relationship has high bias; it misses the true pattern no matter how much data you feed it. An underfitting model is systematically wrong. The cost is predictable error on both training and test data. The benefit is that increasing data or training time cannot save you. You need a more complex model.
Simple models (shallow decision trees, linear classifiers) undercomplicate reality. They miss interactions and curved boundaries. High bias means you're ignoring important patterns in your data.
Overfitting: when the model memorizes noise
Variance measures how much a model's predictions would shift if you retrained it on different data. An overfit model captures random noise in the training set as if it were signal. It performs well on the training data but fails on new data it hasn't seen. The test error is high while training error is low.
The tradeoff appears because simpler models have high bias but low variance (they ignore patterns but don't chase noise). Complex models have low bias but high variance (they fit the true signal but also pick up jitter). The goal is to find the complexity level that minimizes total error (bias squared plus variance). Too little complexity: bias dominates. Too much: variance dominates. Regularization, cross-validation, and early stopping all help navigate this curve.