Huber Loss

Huber Loss is a Loss Function for regression that combines Mean-Squared Error and Mean Absolute Error. For small errors (below a threshold $\delta$) it's quadratic like MSE, and for large errors it's linear like MAE:

$L_\delta(y, \hat{y}) = \begin{cases} \frac{1}{2}(y - \hat{y})^2 & \text{if } |y - \hat{y}| \le \delta \\ \delta \left(|y - \hat{y}| - \frac{1}{2}\delta\right) & \text{otherwise} \end{cases}$

This makes it less sensitive to outliers than MSE, while still being smooth around zero.

It's commonly used in Deep-Q Learning to stop large errors from causing huge, unstable updates.