Home About

Tags

MachineLearning (26) LinearAlgebra (16) GameDesign (12) LargeLanguageModels (11) ComputerScience (11) SoftwareEngineering (11) AudioEngineering (7) DiscreteMath (6) AutomatedTesting (6) ReinforcementLearning (5) Roblox (5) Zettelkasten (5) More

Notes by Lex Toumbourou

Positional Encoding

Apr 21, 2024 permanent

Part of the Transformer architecture. Create an embedding to represent sequence position concatenated to the token embedding.