Home About

Tags

MachineLearning (25) LinearAlgebra (16) GameDesign (13) SoftwareEngineering (11) AudioEngineering (7) AutomatedTesting (6) ComputerScience (5) DiscreteMath (5) Roblox (5) Zettelkasten (5) LearningandTeaching (4) Lua (4) More

Notes by Lex Toumbourou

Mixtral 8x7B

Oct 15, 2024 permanent

A Sparse Mixture of Experts Model model described in Mixtral of Experts (Jan 2024).

Figure 1