Scaled-Dot Product Attention
The specific self-attention formulation from the Transformer paper, distinguished by scaling scores by the square root of the attention dimension.
Notes tagged LargeLanguageModels
-
-
Self-Attention
A method of computing a token representation that includes the context of surrounding tokens.
-
Disputing a Parking Fine with ChatGPT
A use case for ChatGPT
-