Notes by Lex Toumbourou

The concept of hierarchical models in Deep Learning refers to a configuration where models feed output into another in sequence. In general, neural networks are said to be hierarchical architectures, since earlier layers feed outputs into future layers. However, some specific types of hierarchical models exist.

For example, the VALL-E model took a hierarchical approach to modelling audio codes, by predicting later codes from earlier quantisers (see Residual Vector Quantisation).