Image Tokenisation

Image Tokenisation is the process of converting an image into a sequence of tokens that a Transformer can process, similar to Tokenisation for text.

There are two common approaches:

• Split the image into fixed-size patches and project each patch into an embedding. These are continuous tokens.
• Use a learned encoder with a Codebook, like VQ-VAE or VQ-GAN, to map regions of the image to discrete token IDs. This lets image generation be treated like language modelling.

See also Audio Tokenisation.