How Gini impurity guides the split choice
A decision tree works by recursively partitioning the input space into pure regions. At each node, the algorithm tests every possible threshold on every feature and picks the split that most reduces impurity (a measure of how mixed the class labels are in the resulting subsets). Gini impurity is one such measure: it is 0 if all samples belong to one class, and 0.5 for a binary split with equal numbers of each class.
The tree greedy-chooses whichever split minimizes Gini in the child nodes. If feature X at value 3.5 separates reds from blues better than feature Y at value 7.2, the algorithm picks X. This local optimization is fast but not globally optimal; a different first split might have led to a better tree overall. But greedy trees are interpretable and efficient.
Stopping and pruning to avoid memorization
Without restrictions, a tree can split until each leaf is pure (a single class). This overfits. Practical trees use stopping criteria (minimum samples per node, maximum depth) or pruning (removing splits that do not improve held-out accuracy). A tree at depth 3 might generalize better than one at depth 20, even if the deep tree trains with lower error.
The strength of tree-based methods is that each split is interpretable (you can ask why a sample went left at a node), and the method handles both numeric and categorical features naturally. The weakness is variance: small changes in data can drastically change the tree structure. Ensemble methods like random forests address this by averaging predictions across many trees.