Notes by Lex Toumbourou

Articles tagged with InformationTheory

• Cross-Entropy

  Jul 29, 2021 permanent    InformationTheory

  

  Cross-entropy measures the average number of bits required to identify an event if you had a coding scheme optimised for one probability distribution $q …$

  Read More

• Information Entropy

  Jul 28, 2021 permanent    InformationTheory

  

  Entropy is a measure of uncertainty of a random variable's possible outcomes.

  It's highest when there are many equally likely outcomes. As you introduce more …

  Read More