LLM Training Pipeline

A typical pipeline for training Large Language Models.

Steps typically include:

• Pre-training - learning general language patterns from huge raw datasets.
• Post-training like:
  • Supervised Fine-Tuning
  • Reinforcement Learning (RL) (sometimes RLHF)