late-interaction Open weights ST compatible text

lightonai/ColBERT-Zero-unsupervised

Reference: https://huggingface.co/lightonai/ColBERT-Zero-unsupervised
Languages: English
License: apache-2.0
Training code: https://github.com/lightonai/pylate/blob/main/examples/train/ColBERT-zero/unsupervised.py
Training data: https://huggingface.co/datasets/nomic-ai/nomic-embed-unsupervised-data
Adapted from: answerdotai/ModernBERT-base
Trained on: COIRCodeSearchNetRetrievalCodeSearchNetCCRetrievalCodeSearchNetRetrievalQuoraRetrieval

Cite this model

Citation (BibTeX)

@misc{chaffin2026colbertzeropretrainpretraincolbert,
  title         = {ColBERT-Zero: To Pre-train Or Not To Pre-train ColBERT models},
  author        = {Antoine Chaffin and Luca Arnaboldi and Amélie Chatelain and Florent Krzakala},
  year          = {2026},
  eprint        = {2602.16609},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2602.16609},
}

Parameters 149M

Active parameters 110M

Embedding dim 128

Max tokens 8,192

Memory 568 MB

Released 2026-02-19

Openness

6/6

Open weights
Open license
Training code
Training data
Paper
Model card

Benchmark scores

Loading…