Move over the hills and the arrow shows the steepest way up; step against it and you descend. Below, slopes multiply through a chain — that is backpropagation.
move over the hills
–
The gradient
For a function of several inputs, the gradient ∇f is the vector of its slopes, one per input. It points the way the surface rises fastest, and its length is how fast. Perpendicular to it, the ground is level — that is why the arrows cross the contour lines at right angles.
The chain rule, which is backpropagation
When one function feeds another, slopes multiply: d f(g(x))/dx = f′(g(x)) · g′(x). A network is a long chain of such functions; backpropagation is this multiplication run from the loss back to every weight. Below, nudge x and watch the three slopes multiply into one.
Chain rule, live
Evidence. The surface is f(x, y) = sin(1.3x) · cos(1.1y) + 0.08 (x² + y²); its gradient (1.3 cos(1.3x) cos(1.1y) + 0.16x, −1.1 sin(1.3x) sin(1.1y) + 0.16y) is evaluated analytically on the page, and the “check” column shows the same slope estimated by finite differences (h = 10⁻⁴) so the two can be compared. The chain is g(x) = 2x + 1, h(u) = u², f(v) = sin v.
Nine seconds elsewhere
The foundations in learning order, then the series they lead to. Each lands in about nine seconds.