From linear scores to probabilities
Logistic regression is a linear classifier for binary outcomes. It computes a linear combination of features (a dot product of feature vector and learned weights), then squashes the result through the sigmoid function, which maps any real number to a probability between 0 and 1. A raw score of 0 becomes 0.5 probability (maximum uncertainty), negative scores drop below 0.5, and positive scores rise above.
The sigmoid function is S(x) = 1 / (1 + exp(-x)). This smooth S-shape provides two benefits: it bounds predictions to valid probabilities, and its derivative has a simple form, enabling efficient gradient-based learning.
Learning and interpretation
Logistic regression is trained by maximum likelihood (minimizing cross-entropy loss). The resulting weights are directly interpretable: a weight of 0.5 on feature X means a one-unit increase in X multiplies the odds of the positive class by exp(0.5) ≈ 1.65. This interpretability is a major advantage over black-box methods.
Logistic regression assumes a linear decision boundary. It cannot fit nonlinear patterns without feature engineering (interactions, polynomials). But for many problems, a linear boundary is sufficient, and the simplicity, speed, and interpretability of logistic regression make it a strong baseline. It is also resistant to overfitting when regularized (L1 or L2 penalty on weights).