Optimising Computation At The Token-Level
Optimising computation at the token-level
Notes in reference/papers
-
-
Learning to Reason without External Rewards
aka Self-Confidence is All You Need
-
Vibe-Coding Mathematical Discoveries
Using evolutionary algorithms with LLM-coding agents
-
NoProp: Training Neural Networks Without Back-Propagation or Forward-Propagation
an alternative training method to backprop that does local layer learning
-
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
learn to reason without any human-annotated data.
-
Playing Atari with Deep Reinforcement Learning
a classic paper applying neural networks to RL for game playing
-
Large Language Models are Zero-Shot Reasoners (May 2022)
improve zero-shot prompt performance of LLMs by adding “Let’s think step by step” before each answer
-
Neural Machine Translation by Jointly Learning to Align and Translate (Sep 2014)
improve the Encoder/Decoder alignment with an Attention Mechanism
-
Thinking LLMs: General Instruction Following with Thought Generation (Oct 2024)
a prompting and fine-tuning method that enables LLMs to engage in a "thinking" process before generating responses
-
Mixtral of Experts (Jan 2024)
a Sparse Mixture of Experts (SMoE) language model
-
Evaluation of OpenAI o1: Opportunities and Challenges of AGI
a comprehensive evaluation of o1-preview across many tasks and domains.
-
AI Meets the Classroom: When Does ChatGPT Harm Learning?
LLMs can help and also hinder learning outcomes
-
No 'Zero-Shot' Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance
a paper that shows a model needs to see a concept exponentially more times to achieve linear improvements
-
Neural Codec Language Models are Zero-Shot Text-to-Speech Synthesizers
VALL-E can generate speech in anyone's voice with only a 3-second sample of the speaker and some text
-
Large-scale Contrastive Language-Audio Pre-training with Feature Fusion and Keyword-to-Caption Augmentation
Notes from paper Large-scale Contrastive Language-Audio Pre-training with Feature Fusion and Keyword-to-Caption Augmentation by Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, Shlomo Dubnov
-
A Discriminative Feature Learning Approach for Deep Face Recognition
Notes from paper A Discriminative Feature Learning Approach for Deep Face Recognition by Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao
-
ArcFace: Additive Angular Margin Loss for Deep Face Recognition
Notes from paper ArcFace: Additive Angular Margin Loss for Deep Face Recognition by Jiankang Deng, Jia Guo, Niannan Xue, Stefanos Zafeiriou