Mori et al., 1999: Improving Content-Based Image Retrieval with Text

Mori et al., 1999 is an early paper on learning from images paired with text: Image-to-word transformation based on dividing and vector quantizing images with words by Yasuhide Mori, Hironobu Takahashi and Ryuichi Oka.

They explored improving content-based image retrieval by training a model to predict the nouns and adjectives in text documents paired with images.

It's cited as an early precursor in Learning Transferable Visual Models From Natural Language Supervision, the CLIP paper.