Gradient

The Gradient is the generalisation of the Derivative for functions with multiple input variables.

For a function $f(x_1, \ldots, x_n)$, the gradient $\nabla f$ is the vector of its partial derivatives:

$\nabla f = \left( \frac{\partial f}{\partial x_1}, \ldots, \frac{\partial f}{\partial x_n} \right)$

It points in the direction of steepest increase of the function, and its magnitude is how fast the function increases in that direction. That's why Gradient Descent takes steps in the opposite direction to minimise a function.