MTEB logo MTEB
Image-Text, Multilingual icon

Image-Text, Multilingual

Cite this benchmark

@inproceedings{xiao2025mieb,
  author = {Xiao, Chenghao and Chung, Isaac and Kerboua, Imene and Stirling, Jamie and Zhang, Xin and Kardos, M\'arton and Solomatin, Roman and Al Moubayed, Noura and Enevoldsen, Kenneth and Muennighoff, Niklas},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  month = {October},
  pages = {22187-22198},
  title = {MIEB: Massive Image Embedding Benchmark},
  year = {2025},
}

Multilingual image embedding quality across 39 languages, spanning image classification (zero-shot and linear probing), clustering, retrieval, compositionality evaluation, document understanding, visual STS, and CV-centric tasks. Extends MIEB(eng) with multilingual retrieval datasets and the multilingual portions of VisualSTS-b and VisualSTS-16.

Reference paper →
Languages 41
Tasks 130
Task Types 10
Models 0