GPT

GPT (Generative Pre-trained Transformer) is a Language Model architecture from Improving Language Understanding by Generative Pre-Training.

It's a decoder-only Transformer that is first pre-trained on a large corpus of unlabelled text to predict the next token, then Fine-Tuning on specific downstream tasks. The original model was released by OpenAI in 2018, and the approach was scaled up in later models like GPT-2 and GPT-3.