Optimising Computation At The Token-Level
Optimising computation at the token-level
Notes tagged LargeLanguageModels
-
-
Learning to Reason without External Rewards
aka Self-Confidence is All You Need
-
Vibe-Coding Mathematical Discoveries
Using evolutionary algorithms with LLM-coding agents
-
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
learn to reason without any human-annotated data.
-
Temperature Scaling
a parameter that controls how confident Softmax predictions are
-
Few-Shot Knowledge-Distillation
Routes LLM tasks to cheaper or more powerful models based on task novelty.
-
Large Language Models are Zero-Shot Reasoners (May 2022)
improve zero-shot prompt performance of LLMs by adding “Let’s think step by step” before each answer
-
Evaluation of OpenAI o1: Opportunities and Challenges of AGI
a comprehensive evaluation of o1-preview across many tasks and domains.
-
Agentic Reasoning
an approach to utilising LLMs that involve multi-state interactions.
-
Scaled-Dot Product Attention
The specific self-attention formulation from the Transformer paper, distinguished by scaling scores by the square root of the attention dimension.