Sigmoid Function

The Sigmoid function, also known as the Logistic Function, squeezes numbers into a probability-like range between 0 and 1.¹ Used in Binary Classification model architectures to compute loss on discrete labels, that is, labels that are either 1 or 0 (hotdog or not hotdog). The equation is:

$S(x) = \frac{1}{1 + e^{-x}}$

Intuitively, when x is infinity ($e^{-\infty}=0$), the Sigmoid becomes $\frac{1}{1}$ and when x is -infinity ($e^{\infty} = \infty$) the Sigmoid becomes $\frac{1}{inf}$. That means the model is incentivised to output values as high as possible in a positive case, and low for the negative case.

(Fox and Guestrin, 2016)

It is named Sigmoid because of its S-like Function Shape. Its name combines the lowercase sigma character and the suffix -oid, which means similar to.

It can be described and plotted in Python, as follows:

{% notebook permanent/notebooks/sigmoid-function.ipynb %}