Word Tokenisation

Word Tokenisation is (obviously) a type of Tokenisation concerned with converting text into word-level tokens.

For example, I don't like cats. could be split into ["I", "don't", "like", "cats", "."].

Splitting on whitespace is the simplest approach, but it has to deal with punctuation and contractions, and it can't represent words that weren't seen during training (out-of-vocabulary words). That's one reason modern language models use subword tokenisation instead.