All posts

One of the world's most intelligent models is (soon-to-be) open weight.

Language models that manage their own context.

Google's long-awaited new frontier model.

Opus 5.5 quality model that supports audio.

A source-code comparison of eleven agent harnesses and the seven parts they have in common.

New Haiku-class model - this time an actual new Haiku model.

Another insanely good value-for-money offering from OpenAI

Better, faster, and cheaper. Doesn't feel like we're slowing down.

An ablation of planning, tools and context management configurations for coding agents.

A new MCP specification that removes handshakes and mandatory sessions.

A new frontier model from the Qwen team

A new benchmark that tests whether AI agents can follow long company policies during realistic work

An LLM-judge approach that brings interpretability and actionability to your scores.

Ultimate benchmaxxing: hacking Hugging Face for test answers.

Moonshot AI's 2.8T open-weight frontier model for coding and knowledge work.

Now we have GPT Pro at home

A large-scale study on long-horizon document tasks.

The two steps to training modern LLMs.

An agentic framework for end-to-end game creation

Self-generated agent context files don't help.

Curated skills boost agent performance by 16 points; self-generated ones don't help at all.

A new paradigm for single-step generative modelling

ImageNet for Mammography

A breast cancer screening method

a query language for RDF databases

A graph-based data model for the Semantic Web

Describing the 3 types of logical reasoning: Abduction, deduction and induction.

an optimisation technique inspired by natural selection

How LLMs use intermediate reasoning tokens, from chain-of-thought prompting to trained reasoning models, and how this relates to other forms of reasoning.

Optimising computation at the token-level

aka Self-Confidence is All You Need

on John Carmack's Upperbound 25 Talk Notes

Using evolutionary algorithms with LLM-coding agents

an alternative training method to backprop that does local layer learning

learn to reason without any human-annotated data.

a classic paper applying neural networks to RL for game playing

a reinforcement learning algorithm for finding optimal policies

A mathematical framework for modelling decision-making under uncertainty

a data structure where each node contains the hash of its child nodes

a public-key encryption system reliant on the practical difficulty of factorising large numbers

a sub graph of a connected graph that contains all vertices, but no cycles

a distribution-based sorting algorithm that works by dividing elements into buckets

a parameter that controls how confident Softmax predictions are

Routes LLM tasks to cheaper or more powerful models based on task novelty.

improve zero-shot prompt performance of LLMs by adding “Let’s think step by step” before each answer

improve the Encoder/Decoder alignment with an Attention Mechanism

a prompting and fine-tuning method that enables LLMs to engage in a "thinking" process before generating responses

a Sparse Mixture of Experts (SMoE) language model

a comprehensive evaluation of o1-preview across many tasks and domains.

LLMs can help and also hinder learning outcomes

a paper that shows a model needs to see a concept exponentially more times to achieve linear improvements

an approach to utilising LLMs that involve multi-state interactions.

Update the probability of an event based on new evidence

A data visualization that uses squares along a 2D grid for representing proportion.

The specific self-attention formulation from the Transformer paper, distinguished by scaling scores by the square root of the attention dimension.

a sequential data structure where we can add an element to the tail, or remove an element from the head.

a data structure where only the top element is accessible

an algorithm that matches 2-equally sizes groups based on preferences.

a popular divide-and-conquer sorting algorithm

VALL-E can generate speech in anyone's voice with only a 3-second sample of the speaker and some text

A method of computing a token representation that includes the context of surrounding tokens.

the smallest polygon that contains a set of points

a widely-known iterate sorting algorithm

the first step to training a neural network successfully

A tokeniser for audio

An activation function for modelling data with periodicity (repeating patterns)

An activation function that outputs a small value for negative numbers

An activation function that replaces negative values with zero

A measure of the interdependence between software modules

A distinct and identifiable unit of code

How elements of a module relate to each other

Notes from paper Large-scale Contrastive Language-Audio Pre-training with Feature Fusion and Keyword-to-Caption Augmentation by Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, Shlomo Dubnov

a model that can associate textual representations with images.

An extension of propositional logic that involves variables and quantifiers.

A system that deals with statements that are either true or false

Declarative sentences that are either true or false (but not both)

A sound wave visualisation of frequency over time

A measure of how accurately the source signal was digitally represented

Wave of pressure which our ears can perceive

How likely an event is to occur

Tells us the ratio between the sine of an angle and the side opposite it will be constant for all angles in a triangle

a method of solving a problem where a function calls itself

A rule that matches inputs with outputs.

the study of graphs

a function represented by a straight line on a graph

a function represented by a U-shaped curve on a graph

A word numerisation technique that weights terms by their importance to a document.

Cardinality refers to the number of elements a set contains.

Notes from Discrete Mathematics by University of London

A set is a unique collection of well-defined objects.

A technique for computing a weighted combination of a sequence of vectors, originally devised to overcome a bottleneck in seq-to-seq translation models.

Notes from paper A Discriminative Feature Learning Approach for Deep Face Recognition by Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao

Notes from paper ArcFace: Additive Angular Margin Loss for Deep Face Recognition by Jiankang Deng, Jia Guo, Niannan Xue, Stefanos Zafeiriou

A learned vector representation that encodes meaning as a point in space, where similar things end up close together.

A value which describes how much a transformation scales an Eigenvector

A set of vectors whose span doesn't change after a transformation.

Notes from 3Blue1Brown's video series, Essence of Linear Algebra

Since any vectors can be Basis Vectors, it's useful to understand how to translate vectors between bases

An object that describes a point and orientation relative to a BasePart

An operation between two 3d vectors that returns a vector.

Notes from Cross product introduction by Khan Academy.

A measure of how a matrix scales space.

A matrix as a transformation of a space.

The set of vectors that defines space.

In Roblox, a CFrame (coordinate frame) is an object that encodes position and rotation in 3D space.

A measurement of an angle in radiuses

A matrix that reverses a transformation.

A square matrix with 1s on the diagonal and 0s elsewhere: multiplying by it leaves any matrix unchanged.

Notes from Coursera's Mathematics for Machine Learning: Linear Algebra by Imperial College London

The Law of Cosines expresses the relationship between the length of a triangle's sides and one of its angles.

Notes from the Khan Academy video series on the Law of Cosines

Roblox's 3D vector type.

Notes from the Deep Learning for Coders (2020) video series by Jeremy Howard and Sylvain Gugger (fast.ai)

An operation between two matrices that produces a new matrix via row-column dot products.

An operation between 2 vectors that returns a number.

A loss function for binary and multi-label classification, also known as log loss.

Resize images to a larger intermediate size before augmenting.

A loss function for multiclass classification: softmax followed by negative log-likelihood.

A measure of how well one probability distribution encodes events from another.

A measure of uncertainty — highest when all outcomes are equally likely, lowest when one dominates.

a multi-class classification loss function

Converts a vector of numbers into probabilities that sum to 1.

Squeezes any number into a range between 0 and 1.

When production data diverges significantly from the training dataset

A multiplayer architecture where the server is the single source of truth for game state.

When the production data is significantly different from the training distribution.

Metrics are usually a proxy for what we really care about

Notes from Sönke Ahrens' guide to the Zettelkasten note-taking system.

Temporary captures of ideas to be processed into permanent notes.

Notes that capture source material in your own words, as an intermediate step toward permanent notes.

Atomic, self-contained notes that capture a single idea and link to related ones.

arriving at false conclusions by only examining success cases

Building a broad network of concepts across disciplines so ideas can reinforce each other.

Starting with the big picture.

The entry point for a cluster of ideas.

A technique for measuring how different your test set is from your training set.

A single datapoint in a 3d-grid, the word is a combination of "volume" and "pixel"

Promotion means proving you can already do the next job.

The tendency to seek out information that confirms what we already believe.

Your brain consolidates information into long-term memory while it's on break.

The belief that any skill can be mastered through effort, rather than being fixed at birth.

Protecting code against invariant violations by making the wrong thing impossible.

Make the wrong thing impossible.

Tests written without knowledge of the software's internal structure

Khorikov's 4 pillars of good unit tests

A prescripted balance of automated test types

Consistency, availability, and partition tolerance — pick two.

A pattern for structuring unit tests

An automated test which verifies a single unit of behaviour, runs quickly and in isolation

A measure of disorder — the number of possible states a system can be in.

The tendency for codebases to become harder to change as they grow.

Metrics that answer the question : how much of our code is executed by our tests?

Notes from Jesse Schell's comprehensive guide to game design.

A market with a lot of competitiors fighting for dominance

When an object inherits from multiple parents.

OOP in Lua via prototypal inheritance: objects delegate missing field lookups to a prototype table.

Metamethods for intercepting table reads and writes: the foundation of OOP in Lua.

A table attached to another table to define its custom behaviour, via metamethods like __add and __index.

Strong multiplayer communities form when players genuinely need each other to succeed.

A map of what each character thinks of every other character.

A two-axis diagram for visualising character relationships in a story.

A fictional world that can be entered through many different types of media.

The art of guiding players through your experience, while still making them feel in control.

A 32-bit number encoding, either signed (-2B to 2B) or unsigned (0 to 4B).

A game designed to generate a unique story each playthrough rather than follow a scripted one.

A graph of expected player interest over time.

Make players feel in control of their experience.

Functions that can pause and resume execution, similar to Python generators.

When tuning game values, double or halve them instead of nudging: it makes the effect obvious.

A game economy is meaningful when spending and earning money form a rewarding loop.

Complex interactions that arise naturally from a simple ruleset.

Strategic actions that aren't in the rules but arise naturally from playing the game.

On the power of unpredictable rewards.

Addictive games keep players exercising a skill, taking risks, and working a strategy simultaneously.

Plan to ship at half the budget, use the rest to make it great.

Define the problem clearly before designing.

Choices only matter if the options are meaningfully different and none is obviously dominant.

Visualising Newton's 2nd law and opposing forces using Roblox's BodyForce object.

A linear story interrupted by gameplay segments.

High-risk, high-reward choices that make games fun.

Every element of a game should reinforce a single theme.

Adding two vectors by summing their corresponding components.

Subtracting two vectors by subtracting their corresponding components.

The more times you test and improve your design, the better your game will be.

deliver working software frequently

The sum of weighted outcomes divided by total outcomes

The tendency to prefer avoiding losses over acquiring equivalent gains.

An object at rest stays at rest, F=ma, and every action has an equal and opposite reaction.

Complete immersion in an activity with energised focus — also known as "in the zone".

Clear goals, immediate feedback, and continuous challenge pitched between boredom and frustration.

An ordered list of numbers that describe an arrow in space.

Multiplying each value in a vector by the same value

A method of detecting nearby objects in 3D space.

A rendering technique that casts rays for each vertical slice of the screen.

Notes from Khan Academy video Basic geometry: language and labels

A line-like figure that continue indefinitely in one direction

Course notes from fastai's Deep Learning 2018

a technique for adding class imbalance by creating synthetic examples

an unsupervised clustering algorithm

a key-value pair data structure with constant time data access

Course notes from Data Structures and Performance on Coursera

The average absolute difference between predictions and labels, less sensitive to outliers than RMSE.

Notes from Machine Learning: Regression by University of Washington on Coursera.

A regression metric that penalises large errors.

a subspace where the dimension is one less than the ambient space

Notes from Coursera's Machine Learning Foundations: A Case Study Approach by Emily Fox and Carlos Guestrin

a technique for reducing the dimensionality of data

An elegant algorithm for finding the shortest path between nodes in a graph.

Notes from Statistics (st095) by Udacity

A system call that duplicates the current process, creating a parent and child that continue from the same point.

A child process whose parent has exited.

A dead process waiting to be reaped.