Domain Shift
When production data diverges significantly from the training dataset
MachineLearningFailureModes
-
-
Out-of-Domain
When the production data is significantly different from the training distribution.
-
Proxy Metrics
Metrics are usually a proxy for what we really care about