Definition
Gradient descent is the step-by-step method an AI uses to gradually correct its own mistakes by adjusting its internal settings until its predictions become as accurate as possible.[1]
At a glance
- It is how an AI model learns: it measures how wrong it is, then nudges its settings to be a little less wrong, over and over.[4]
- The learning rate is the step size. Too big and it overshoots the answer; too small and training takes forever and costs more.[2]
- It can get stuck in a “good enough” valley that is not the best possible answer, which is why model quality varies.[3]
- Nearly every modern AI tool, from chatbots to fraud detection, is trained this way.[1]
Why it matters to your business
Gradient descent is the engine behind every AI product you might buy or build. Its settings directly affect two things you care about: how much training costs (more steps means more compute spend) and how accurate the final model is. Vendors who tune it well ship cheaper, sharper models.[4]
The hidden trade-off
Training is a balancing act. Rush it with big steps and the model never settles on a good answer. Crawl with tiny steps and you burn time and money.[2] The model can also settle into a mediocre “valley” that looks done but is not optimal, so results are never fully guaranteed.[3]
Bottom line
Gradient descent is the patient, repeat-until-right learning process that turns a raw AI model into one that actually makes useful predictions.