late-interaction Open weights ST compatible text
lightonai/mLateOn-unsupervised
- Languages
- ArabicEnglishFrenchGermanItalianNorwegian BokmålPortugueseSpanishSwedish
- License
- apache-2.0
- Training code
- https://github.com/lightonai/mdenseon-mlateon/blob/main/scripts/pretrain/multilingual_late_interaction.py
- Training data
- https://huggingface.co/datasets/lightonai/multilingual-embeddings-pre-training-curated
- Adapted from
- jhu-clsp/mmBERT-base
- Trained on
- CQADupstackRetrievalDBPediaQuoraRetrievalSCIDOCSSciFact
Cite this model
Citation (BibTeX)
@misc{sourty2026denseonlateonfullyopen,
title = {DenseOn with the LateOn: Fully Open Dense and Late-Interaction Models for Multilingual, Long-Context, and Code Search},
author = {Raphaël Sourty and Antoine Chaffin and Paulo Roberto Moura Junior and Amélie Chatelain},
year = {2026},
eprint = {2607.27178},
archivePrefix = {arXiv},
primaryClass = {cs.CL},
url = {https://arxiv.org/abs/2607.27178},
}Parameters 307M
Active parameters 110M
Embedding dim 128
Max tokens 8,192
Memory 585 MB
Released 2026-07-30
Openness
6/6
- Open weights
- Open license
- Training code
- Training data
- Paper
- Model card