VQ-GAN

VQ-GAN is an image model from Taming Transformers for High-Resolution Image Synthesis that combines the discrete codebook of VQ-VAE with adversarial training from Generative Adversarial Network.

A convolutional encoder compresses an image into a small grid of codebook indices, and the decoder is trained with a perceptual loss and a patch-based discriminator, so it can reconstruct sharp images from a highly compressed code. A Transformer is then trained autoregressively over the code indices to generate high-resolution images. See also Improved VQGAN.