Jagged Frontier of LLM Capability
The Jagged Frontier is the observation that LLMs can be strong in some tasks while making severe errors in others (DellAcqua et al., 2023)
A recent paper LLMs Corrupt Your Documents When You Delegate showed how through a multi-step backtranslation-inspired approach, they demonstrate that LLMs had severe issues with degradation across all document domains except Python.