Calculus for Machine Learning

A derivative describes local change; a gradient collects partial derivatives of a scalar function with respect to several variables. Machine learning combines many such functions into a computational graph. Applying the chain rule through that graph makes it possible to calculate how each parameter contributes to the final loss. Calculus also describes curvature, local approximations and the distinction between a stationary point and a minimum. The practical scope includes multivariable differentiation, vector-valued transformations and reasoning about smoothness. Automatic differentiation performs the bookkeeping, but does not choose a suitable objective or establish that the result represents the intended problem.

A practitioner should be able to derive the gradient of a simple objective, follow how tensor operations compose and identify where a transformation prevents gradient flow. Important decisions include reduction over examples, treatment of constants and the scale of competing loss terms. For a custom operation, compare analytical or automatic gradients with a small numerical check away from discontinuities. The result is an objective whose updates can be explained, together with evidence that the implementation differentiates the intended quantity rather than an accidental reshaping or detached intermediate.

A gradient gives local information and does not guarantee a globally best model. Numerical differences can be unreliable with poorly chosen step sizes, floating-point noise or nonsmooth operations. Functions such as maximum and absolute value need careful treatment at their corners, and discrete choices cannot generally be differentiated directly. Backpropagation is an application of the chain rule, not a separate justification for the objective. Check finite values, gradient magnitude and dependency paths before interpreting a stalled training run as a lack of useful data.