All Notes

a distribution-based sorting algorithm that works by dividing elements into buckets

a parameter that controls how confident Softmax predictions are

Routes LLM tasks to cheaper or more powerful models based on task novelty.

improve zero-shot prompt performance of LLMs by adding “Let’s think step by step” before each answer

improve the Encoder/Decoder alignment with an Attention Mechanism

a prompting and fine-tuning method that enables LLMs to engage in a "thinking" process before generating responses

a Sparse Mixture of Experts (SMoE) language model

a comprehensive evaluation of o1-preview across many tasks and domains.

LLMs can help and also hinder learning outcomes

System 2 thinking is characterised by slow, deliberate, and logical reasoning, requiring conscious effort and attention to solve complex problems. Unlike the intuitive nature of …