Lesson 18 of 20 · about 17 min

Gradient descent

Setting both partial derivatives to zero works for the pool site, but a model with a million settings has a million equations. Instead, start anywhere and keep stepping downhill along minus the gradient. That simple loop, gradient descent, trains almost every modern AI. You'll tune the step size until it neither crawls nor explodes, watch it zig-zag down a narrow valley, and see it get stuck in the wrong hollow.