Home /permanent

Log Loss

Log Loss, or logistic loss or cross-entropy loss - is a specific case of Negative Log-Likelihood for binary classification problems.

The formula for a single data point is: −(y×log⁡(y^)+(1−y)×log⁡(1−y^))-(y \times \log(\hat{y}) + (1 - y) \times \log(1 - \hat{y})) which is equivalent to:

{−log⁡(y^)if y=1−log⁡(1−y^)if y=0 \begin{cases} -\log(\hat{y}) & \text{if } y = 1\\ -\log(1 - \hat{y}) & \text{if } y = 0 \end{cases}

To calculate the log loss for an entire dataset, you take the average of each datapoint: LogLoss=−1n∑(y×log⁡(y^)+(1−y)×log⁡(1−y^))LogLoss = -\frac{1}{n} \sum (y \times \log(\hat{y}) + (1 - y) \times \log(1 - \hat{y}))

Log Loss is the same as negative log-likelihood after converting binary into multi-class by one-hot encoding the binary labels.

Since the log of a value between 0 and 1 is negative, we add the negative sign to convert it into a positive number.