Large Language Models
Large Language Models or LLMs are a type of Language Model, typically a Transformer trained on a next-word prediction task on a massive dataset with a high parameter count.
Training usually happens in stages. First, Pretraining on huge amounts of text produces a base model that's good at continuing text. Then post-training, such as Supervised Fine-Tuning and Reinforcement Learning from Human Feedback, turns it into an assistant that follows instructions (Ouyang et al., 2022).
GPT-3 showed that at a large enough scale, models can learn new tasks from just a few examples in the prompt, without any fine-tuning. This is known as In-Context Learning (Brown et al., 2020). More recently, models are also trained to reason before they answer: see LLM Reasoning.
References
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language Models are Few-Shot Learners. 2020. doi:10.48550/ARXIV.2005.14165. ↩
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. March 2022. arXiv:2203.02155, doi:10.48550/arXiv.2203.02155. ↩