Jagged Frontier of LLM Capability
The observation that LLMs can be strong in some tasks while making errors in others
The Jagged Frontier is the observation that LLMs can be strong in some tasks while making severe errors in others DellAcqua et al. (2023)
A recent paper LLMs Corrupt Your Documents When You Delegate showed how through a multi-step backtranslation-inspired approach, they demonstrate that LLMs had severe issues with degradation across all document domains except Python.