Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Paper โข 1908.10084 โข Published โข 19
CrossEncoder based on BAAI/bge-reranker-v2-m3
This is a Cross Encoder model finetuned from BAAI/bge-reranker-v2-m3 using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
Model Details
Model Description
- Model Type: Cross Encoder
- Base model: BAAI/bge-reranker-v2-m3
- Maximum Sequence Length: 8192 tokens
- Number of Output Labels: 1 label
Model Sources
- Documentation: Sentence Transformers Documentation
- Documentation: Cross Encoder Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Cross Encoders on Hugging Face
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import CrossEncoder
# Download from the ๐ค Hub
model = CrossEncoder("cross_encoder_model_id")
# Get scores for pairs of texts
pairs = [
["What is the significance of Samsung Electronics as a Korean brand in the list of the world's top 100 trademarks?", '็ฑไบๅ
ถๆญฃๅคไบไบงๅๅผๅไธ้ช่ฏๆๅ
ฅ้ถๆฎต๏ผๅฝฑๅไบๅ
ฌๅธ็ๆ่ตๆถ็ใ\n\n\u3000\u3000่ฎพๅคไผไธ๏ผ\n\n\n\u3000\u3000ไธ็ปฉ็ฟปๅๅข้ฟ\n\n\u3000\u3000่ฝ็ถๆดไฝๅๅฏผไฝๆฟๅๅฐๆช่ตฐๅบไฝ่ฐท๏ผไฝๅฝไบงๆฟไปฃ้ๆฑๆจๅจไธ๏ผ่ฎพๅค็ฏ่ไผไธไฟๆ้ๅจๆ้ซ้ๅข้ฟ๏ผ้พๅคด่ฎพๅคๅๅไธๅๅนดไธ็ปฉ็ฟปๅๅข้ฟใๅฝๅฎถ็ป่ฎกๅฑๆๆฐๆซ้ฒ๏ผๅด็ป็ๅ
ๆโๅก่ๅญโๅทฅ็จ๏ผไปๅนดไธๅๅนดๅๅฏผไฝ็ธๅ
ณ่กไธๅถ้ ไธๅข้ฟ่พๅฟซ๏ผๅๅฏผไฝๅจไปถไธ็จ่ฎพๅคๅถ้ ไธๅขๅ ๅผๅข้ฟ30.9%ใ'],
['ๆ นๆฎๆไธญๆๅฐ็ไธๆธธใไธญๆธธๅไธๆธธ็ไธๅ็ฏ่๏ผ่ฏท็ฎ่ฆๆ่ฟฐๅๅฏผไฝไบงไธ้พ็ๆดไฝ็ปๆใ', 'DRAMๅธๅบ็ฑไธๆใ็พๅ
ใๆตทๅๅฃซๅๆญไบ95%็ไปฝ้ข๏ผ็ฎๅๅฝไบงๅๅๅ่ฅ้ฟ้ซๅทฒ็ปๅผๅง้ไบงๅนถๅจๅฎ็ฝไธๆถไบ็ธๅ
ณไบงๅ๏ผ็ดซๅ
้ๅขไนๅทฒๅปบ็ซDRAMไบไธ้จๅๅคๅปบๅใ\n\nNAND Flash็ๅธๅบ็ฑไธๆใ่ฅฟๆฐใ้ ไพ ็ญ6ๅฎถไผไธๅๆญใ็ฎๅNAND Flash็ๅๅฑๆนๅๆฏ3Dๅ ๅ ๏ผๅฝๅคๅ
่ฟไผไธๅๅทฒ็บท็บทๅผๅๅบ100ๅฑไปฅไธๅ ๅ ็NAND Flashใๅฝไบงๅๅ้ฟๆฑๅญๅจๅทฒๅฎฃๅธ128ๅฑไบงๅ็ ๅๆๅ๏ผไธๅฝๅคๅ
่ฟไผไธ็ๅทฎ่ท่ถๆฅ่ถๅฐ๏ผๅทฒๆไธบๅญๅจๅฝไบง่ชไธปๅ็ไธญๅๅ้ใ'],
['ๆ นๆฎไธไธๆไฟกๆฏ๏ผๆๅบไธไธช้ฎ้ขใ', 'ๅๅฏผไฝๆๆๆฏๅถไฝๆถไฝ็ฎกใ้ๆ็ต่ทฏใๅ
็ตๅญๅจไปถ็้่ฆๆๆใ\n\nๆ็
งๅๅญฆ็ปๆไธๅ๏ผๅๅฏผไฝๆๆๅฏไปฅๅไธบๅ
็ด ๅๅฏผไฝๅๅๅ็ฉๅๅฏผไฝไธคๅคง็ฑปใ'],
['What is the projected annual growth rate of the automotive semiconductor market from 2013 to 2018 according to IHS data?', '้ฟ็ต็งๆไฝไธบA่กๅๅฏผไฝๅฐ่ฃ
ๆต่ฏ้พๅคด๏ผ็ฌฌไบๅญฃๅบฆไธ็ปฉไน็ฏๆฏๅคงๅน
ๅข้ฟใไธ็ปฉ้ขๅๆพ็คบ๏ผไปๅนดไธๅๅนดๅ
ฌๅธๅฎ็ฐๅฝๆฏๅๅฉๆถฆไธบ4.46ไบฟๅ
ๅฐ5.46ไบฟๅ
๏ผๅๆฏๅๅฐ64.65%ๅฐ71.08%ใๅ
ฌๅธไธๅญฃๅบฆๅฎ็ฐๅฝๆฏๅๅฉๆถฆ็บฆ1.1ไบฟๅ
๏ผ็ฌฌไบๅญฃๅบฆๆๅฎ็ฐ็ๅฉ3.36ไบฟ่ณ4.36ไบฟๅ
๏ผ็ฏๆฏไธๅญฃๅบฆๅข้ฟ็บฆไธคๅไปฅไธ๏ผๅ
ฌๅธไธๆญๆๅ
ฅๆฑฝ่ฝฆ็ตๅญใๅทฅไธ็ตๅญๅ้ซๆง่ฝ่ฎก็ฎ็ญ้ขๅ๏ผไธบๆฐไธ่ฝฎๅบ็จ้ๆฑๅข้ฟๅๅฅฝๅๅคใๆญคๅ๏ผ้ฟ็ต็งๆไป็ป๏ผ้ขๅ้ซ็ฎๅ่ฏ็ๅ
ฌๅธๆจๅบไบChiplet้ซๆง่ฝๅฐ่ฃ
ๆๆฏๅนณๅฐXDFOIใ'],
['ไฝ ่ฎคไธบไบบๅทฅๆบ่ฝๆชๆฅๅฏ่ฝๅจๅชไบ้ขๅๅๆฅไฝ็จ๏ผ', '98ไบฟๅ
๏ผๅ
ถไธญ๏ผๅฝๆๆฑๅ
ๆๅคฑ้ ๆๅๅฉๆถฆๅๅฐ็บฆ2.03ไบฟๅ
๏ผๅ้ค่ฏฅๅ ็ด ๏ผไธๅๅนดๅ
ฌๅธๅๅฉๆถฆไธบๆญฃใ้ๅฏๅพฎ็ตไป็ป๏ผๅ
จ็ๅๅฏผไฝๅธๅบ็ฒ่ฝฏ,ไธๆธธ้ๆฑๅค่ไธๅ้ขๆ๏ผๅฏผ่ดๅฐๆต็ฏ่ไธๅกๆฟๅ,ๅ
ฌๅธไผ ็ปไธๅกไบฆๅๅฐ่พๅคงๅฝฑๅใไฝไธบๅบๅฏน๏ผๅ
ฌๅธ่ฐๆดไบงๅๅธๅฑ๏ผๅจ้ซๆง่ฝ่ฎก็ฎใๆฐ่ฝๆบใๆฑฝ่ฝฆ็ตๅญใๅญๅจใๆพ็คบ้ฉฑๅจ็ญ้ขๅๅฎ็ฐ่ฅๆถๅข้ฟ๏ผ็งฏๆๆจๅจChiplet๏ผ่ฏ็ฒ๏ผๅธๅบๅๅบ็จ,ๅฎ็ฐไบ่งๆจกๆง้ไบงใ'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)
# Or rank different texts based on similarity to a single text
ranks = model.rank(
"What is the significance of Samsung Electronics as a Korean brand in the list of the world's top 100 trademarks?",
[
'็ฑไบๅ
ถๆญฃๅคไบไบงๅๅผๅไธ้ช่ฏๆๅ
ฅ้ถๆฎต๏ผๅฝฑๅไบๅ
ฌๅธ็ๆ่ตๆถ็ใ\n\n\u3000\u3000่ฎพๅคไผไธ๏ผ\n\n\n\u3000\u3000ไธ็ปฉ็ฟปๅๅข้ฟ\n\n\u3000\u3000่ฝ็ถๆดไฝๅๅฏผไฝๆฟๅๅฐๆช่ตฐๅบไฝ่ฐท๏ผไฝๅฝไบงๆฟไปฃ้ๆฑๆจๅจไธ๏ผ่ฎพๅค็ฏ่ไผไธไฟๆ้ๅจๆ้ซ้ๅข้ฟ๏ผ้พๅคด่ฎพๅคๅๅไธๅๅนดไธ็ปฉ็ฟปๅๅข้ฟใๅฝๅฎถ็ป่ฎกๅฑๆๆฐๆซ้ฒ๏ผๅด็ป็ๅ
ๆโๅก่ๅญโๅทฅ็จ๏ผไปๅนดไธๅๅนดๅๅฏผไฝ็ธๅ
ณ่กไธๅถ้ ไธๅข้ฟ่พๅฟซ๏ผๅๅฏผไฝๅจไปถไธ็จ่ฎพๅคๅถ้ ไธๅขๅ ๅผๅข้ฟ30.9%ใ',
'DRAMๅธๅบ็ฑไธๆใ็พๅ
ใๆตทๅๅฃซๅๆญไบ95%็ไปฝ้ข๏ผ็ฎๅๅฝไบงๅๅๅ่ฅ้ฟ้ซๅทฒ็ปๅผๅง้ไบงๅนถๅจๅฎ็ฝไธๆถไบ็ธๅ
ณไบงๅ๏ผ็ดซๅ
้ๅขไนๅทฒๅปบ็ซDRAMไบไธ้จๅๅคๅปบๅใ\n\nNAND Flash็ๅธๅบ็ฑไธๆใ่ฅฟๆฐใ้ ไพ ็ญ6ๅฎถไผไธๅๆญใ็ฎๅNAND Flash็ๅๅฑๆนๅๆฏ3Dๅ ๅ ๏ผๅฝๅคๅ
่ฟไผไธๅๅทฒ็บท็บทๅผๅๅบ100ๅฑไปฅไธๅ ๅ ็NAND Flashใๅฝไบงๅๅ้ฟๆฑๅญๅจๅทฒๅฎฃๅธ128ๅฑไบงๅ็ ๅๆๅ๏ผไธๅฝๅคๅ
่ฟไผไธ็ๅทฎ่ท่ถๆฅ่ถๅฐ๏ผๅทฒๆไธบๅญๅจๅฝไบง่ชไธปๅ็ไธญๅๅ้ใ',
'ๅๅฏผไฝๆๆๆฏๅถไฝๆถไฝ็ฎกใ้ๆ็ต่ทฏใๅ
็ตๅญๅจไปถ็้่ฆๆๆใ\n\nๆ็
งๅๅญฆ็ปๆไธๅ๏ผๅๅฏผไฝๆๆๅฏไปฅๅไธบๅ
็ด ๅๅฏผไฝๅๅๅ็ฉๅๅฏผไฝไธคๅคง็ฑปใ',
'้ฟ็ต็งๆไฝไธบA่กๅๅฏผไฝๅฐ่ฃ
ๆต่ฏ้พๅคด๏ผ็ฌฌไบๅญฃๅบฆไธ็ปฉไน็ฏๆฏๅคงๅน
ๅข้ฟใไธ็ปฉ้ขๅๆพ็คบ๏ผไปๅนดไธๅๅนดๅ
ฌๅธๅฎ็ฐๅฝๆฏๅๅฉๆถฆไธบ4.46ไบฟๅ
ๅฐ5.46ไบฟๅ
๏ผๅๆฏๅๅฐ64.65%ๅฐ71.08%ใๅ
ฌๅธไธๅญฃๅบฆๅฎ็ฐๅฝๆฏๅๅฉๆถฆ็บฆ1.1ไบฟๅ
๏ผ็ฌฌไบๅญฃๅบฆๆๅฎ็ฐ็ๅฉ3.36ไบฟ่ณ4.36ไบฟๅ
๏ผ็ฏๆฏไธๅญฃๅบฆๅข้ฟ็บฆไธคๅไปฅไธ๏ผๅ
ฌๅธไธๆญๆๅ
ฅๆฑฝ่ฝฆ็ตๅญใๅทฅไธ็ตๅญๅ้ซๆง่ฝ่ฎก็ฎ็ญ้ขๅ๏ผไธบๆฐไธ่ฝฎๅบ็จ้ๆฑๅข้ฟๅๅฅฝๅๅคใๆญคๅ๏ผ้ฟ็ต็งๆไป็ป๏ผ้ขๅ้ซ็ฎๅ่ฏ็ๅ
ฌๅธๆจๅบไบChiplet้ซๆง่ฝๅฐ่ฃ
ๆๆฏๅนณๅฐXDFOIใ',
'98ไบฟๅ
๏ผๅ
ถไธญ๏ผๅฝๆๆฑๅ
ๆๅคฑ้ ๆๅๅฉๆถฆๅๅฐ็บฆ2.03ไบฟๅ
๏ผๅ้ค่ฏฅๅ ็ด ๏ผไธๅๅนดๅ
ฌๅธๅๅฉๆถฆไธบๆญฃใ้ๅฏๅพฎ็ตไป็ป๏ผๅ
จ็ๅๅฏผไฝๅธๅบ็ฒ่ฝฏ,ไธๆธธ้ๆฑๅค่ไธๅ้ขๆ๏ผๅฏผ่ดๅฐๆต็ฏ่ไธๅกๆฟๅ,ๅ
ฌๅธไผ ็ปไธๅกไบฆๅๅฐ่พๅคงๅฝฑๅใไฝไธบๅบๅฏน๏ผๅ
ฌๅธ่ฐๆดไบงๅๅธๅฑ๏ผๅจ้ซๆง่ฝ่ฎก็ฎใๆฐ่ฝๆบใๆฑฝ่ฝฆ็ตๅญใๅญๅจใๆพ็คบ้ฉฑๅจ็ญ้ขๅๅฎ็ฐ่ฅๆถๅข้ฟ๏ผ็งฏๆๆจๅจChiplet๏ผ่ฏ็ฒ๏ผๅธๅบๅๅบ็จ,ๅฎ็ฐไบ่งๆจกๆง้ไบงใ',
]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
Evaluation
Metrics
Cross Encoder Reranking
- Dataset:
train-eval - Evaluated with
CERerankingEvaluatorwith these parameters:{ "at_k": 10 }
| Metric | Value |
|---|---|
| map | 0.9177 |
| mrr@10 | 0.9177 |
| ndcg@10 | 0.9377 |
Training Details
Training Dataset
Unnamed Dataset
- Size: 890 training samples
- Columns:
sentence_0,sentence_1, andlabel - Approximate statistics based on the first 890 samples:
sentence_0 sentence_1 label type string string int details - min: 13 characters
- mean: 55.08 characters
- max: 237 characters
- min: 64 characters
- mean: 179.63 characters
- max: 249 characters
- 0: ~80.00%
- 1: ~20.00%
- Samples:
sentence_0 sentence_1 label What is the significance of Samsung Electronics as a Korean brand in the list of the world's top 100 trademarks?็ฑไบๅ ถๆญฃๅคไบไบงๅๅผๅไธ้ช่ฏๆๅ ฅ้ถๆฎต๏ผๅฝฑๅไบๅ ฌๅธ็ๆ่ตๆถ็ใ
ใใ่ฎพๅคไผไธ๏ผ
ใใไธ็ปฉ็ฟปๅๅข้ฟ
ใใ่ฝ็ถๆดไฝๅๅฏผไฝๆฟๅๅฐๆช่ตฐๅบไฝ่ฐท๏ผไฝๅฝไบงๆฟไปฃ้ๆฑๆจๅจไธ๏ผ่ฎพๅค็ฏ่ไผไธไฟๆ้ๅจๆ้ซ้ๅข้ฟ๏ผ้พๅคด่ฎพๅคๅๅไธๅๅนดไธ็ปฉ็ฟปๅๅข้ฟใๅฝๅฎถ็ป่ฎกๅฑๆๆฐๆซ้ฒ๏ผๅด็ป็ๅ ๆโๅก่ๅญโๅทฅ็จ๏ผไปๅนดไธๅๅนดๅๅฏผไฝ็ธๅ ณ่กไธๅถ้ ไธๅข้ฟ่พๅฟซ๏ผๅๅฏผไฝๅจไปถไธ็จ่ฎพๅคๅถ้ ไธๅขๅ ๅผๅข้ฟ30.9%ใ0ๆ นๆฎๆไธญๆๅฐ็ไธๆธธใไธญๆธธๅไธๆธธ็ไธๅ็ฏ่๏ผ่ฏท็ฎ่ฆๆ่ฟฐๅๅฏผไฝไบงไธ้พ็ๆดไฝ็ปๆใDRAMๅธๅบ็ฑไธๆใ็พๅ ใๆตทๅๅฃซๅๆญไบ95%็ไปฝ้ข๏ผ็ฎๅๅฝไบงๅๅๅ่ฅ้ฟ้ซๅทฒ็ปๅผๅง้ไบงๅนถๅจๅฎ็ฝไธๆถไบ็ธๅ ณไบงๅ๏ผ็ดซๅ ้ๅขไนๅทฒๅปบ็ซDRAMไบไธ้จๅๅคๅปบๅใ
NAND Flash็ๅธๅบ็ฑไธๆใ่ฅฟๆฐใ้ ไพ ็ญ6ๅฎถไผไธๅๆญใ็ฎๅNAND Flash็ๅๅฑๆนๅๆฏ3Dๅ ๅ ๏ผๅฝๅคๅ ่ฟไผไธๅๅทฒ็บท็บทๅผๅๅบ100ๅฑไปฅไธๅ ๅ ็NAND Flashใๅฝไบงๅๅ้ฟๆฑๅญๅจๅทฒๅฎฃๅธ128ๅฑไบงๅ็ ๅๆๅ๏ผไธๅฝๅคๅ ่ฟไผไธ็ๅทฎ่ท่ถๆฅ่ถๅฐ๏ผๅทฒๆไธบๅญๅจๅฝไบง่ชไธปๅ็ไธญๅๅ้ใ0ๆ นๆฎไธไธๆไฟกๆฏ๏ผๆๅบไธไธช้ฎ้ขใๅๅฏผไฝๆๆๆฏๅถไฝๆถไฝ็ฎกใ้ๆ็ต่ทฏใๅ ็ตๅญๅจไปถ็้่ฆๆๆใ
ๆ็ งๅๅญฆ็ปๆไธๅ๏ผๅๅฏผไฝๆๆๅฏไปฅๅไธบๅ ็ด ๅๅฏผไฝๅๅๅ็ฉๅๅฏผไฝไธคๅคง็ฑปใ0 - Loss:
BinaryCrossEntropyLosswith these parameters:{ "activation_fn": "torch.nn.modules.linear.Identity", "pos_weight": null }
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: stepsnum_train_epochs: 2fp16: True
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: stepsprediction_loss_only: Trueper_device_train_batch_size: 8per_device_eval_batch_size: 8per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 1eval_accumulation_steps: Nonetorch_empty_cache_steps: Nonelearning_rate: 5e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1num_train_epochs: 2max_steps: -1lr_scheduler_type: linearlr_scheduler_kwargs: {}warmup_ratio: 0.0warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Falsefp16: Truefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Falseignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torchoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Nonehub_always_push: Falsehub_revision: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseinclude_for_metrics: []eval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falseuse_liger_kernel: Falseliger_kernel_config: Noneeval_use_gather_object: Falseaverage_tokens_across_devices: Falseprompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}
Training Logs
| Epoch | Step | train-eval_ndcg@10 |
|---|---|---|
| 0.8929 | 100 | 0.9377 |
Framework Versions
- Python: 3.9.20
- Sentence Transformers: 5.0.0
- Transformers: 4.53.1
- PyTorch: 2.4.1
- Accelerate: 1.8.1
- Datasets: 3.6.0
- Tokenizers: 0.21.2
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
- Downloads last month
- 25
Model tree for Setsuna007/ft-bge-reranker-v2-m3-test
Base model
BAAI/bge-reranker-v2-m3Paper for Setsuna007/ft-bge-reranker-v2-m3-test
Evaluation results
- Map on train evalself-reported0.918
- Mrr@10 on train evalself-reported0.918
- Ndcg@10 on train evalself-reported0.938