Domain Shift

When the data your model sees has diverged significantly from the training dataset, it's called a Domain Shift.

Howard et al. (2020) (pg. 104)

In the 2019 Data Science Bowl, PBS Kids ran a Kaggle competition to determine how well a player would do on a challenge based on their behavior so far. Because the training data was particular to the current version of the game, including very level-specific data like the coordinates of mouse clicks, the models would be sensitive to tiny changes made to the game. Changing the order of levels or moving a sprite could trigger a significant domain shift, making the current production data effectively Out-of-Domain.

Cover New Vans Vs. Old Vans by Danny Lopez on Flickr.


Jeremy Howard, Sylvain Gugger, and Soumith Chintala. Deep Learning for Coders with Fastai and PyTorch: AI Applications without a PhD. O'Reilly Media, Inc., Sebastopol, California, 2020. ISBN 978-1-4920-4552-6.