Ran MTEB evaluation on Bulgarian tasks comparing EmbeddingGemma-300M (
text
Ran MTEB evaluation on Bulgarian tasks comparing EmbeddingGemma-300M (
https://huggingface.co/google/embeddinggemma-300m)
google/embeddinggemma-300m
model
resource
https://huggingface.co/google/embeddinggemma-300m)
) vs Multilingual-E5-Large (
text
) vs Multilingual-E5-Large (
https://huggingface.co/intfloat/multilingual-e5-large
intfloat/multilingual-e5-large
model
resource
https://huggingface.co/intfloat/multilingual-e5-large
)
text
)
new_line
new_line
EmbeddingGemma-300M scored 71.6% average while E5-Large got 75.9%. Pretty solid results for EmbeddingGemma considering it's half the size and uses way less resources.
text
EmbeddingGemma-300M scored 71.6% average while E5-Large got 75.9%. Pretty solid results for EmbeddingGemma considering it's half the size and uses way less resources.
new_line
new_line
EmbeddingGemma actually beats E5-Large on sentiment analysis and natural language inference. E5-Large wins on retrieval and bitext mining tasks.
text
EmbeddingGemma actually beats E5-Large on sentiment analysis and natural language inference. E5-Large wins on retrieval and bitext mining tasks.
new_line
new_line
The 300M model has 4x longer context window (2048 vs 512 tokens) and lower carbon footprint which is good. 
text
The 300M model has 4x longer context window (2048 vs 512 tokens) and lower carbon footprint which is good. 
new_line
new_line
Both models work great for Bulgarian but have different strengths depending what you need. 
text
Both models work great for Bulgarian but have different strengths depending what you need. 
new_line
new_line
Blog article about the usage: 
text
Blog article about the usage: 
https://huggingface.co/blog/embeddinggemma
https://huggingface.co/blog/embeddinggemma
link
text
new_line
new_line
PS: Don't forget to use the recommended libraries versions :D
text
PS: Don't forget to use the recommended libraries versions :D
new_line
new_line
pip install git+https://github.com/huggingface/transformers@v4.56.0-Embedding-Gemma-preview
pip install sentence-transformers>=5.0.0
```
pip install git+https://github.com/huggingface/transformers@v4.56.0-Embedding-Gemma-preview
pip install sentence-transformers>=5.0.0
```
code_fence
new_line
Ran MTEB evaluation on Bulgarian tasks comparing EmbeddingGemma-300M (https://huggingface.co/google/embeddinggemma-300m)) vs Multilingual-E5-Large (https://huggingface.co/intfloat/multilingual-e5-large)

EmbeddingGemma-300M scored 71.6% average while E5-Large got 75.9%. Pretty solid results for EmbeddingGemma considering it's half the size and uses way less resources.

EmbeddingGemma actually beats E5-Large on sentiment analysis and natural language inference. E5-Large wins on retrieval and bitext mining tasks.

The 300M model has 4x longer context window (2048 vs 512 tokens) and lower carbon footprint which is good. 

Both models work great for Bulgarian but have different strengths depending what you need. 

Blog article about the usage: https://huggingface.co/blog/embeddinggemma 

PS: Don't forget to use the recommended libraries versions :D

```
pip install git+https://github.com/huggingface/transformers@v4.56.0-Embedding-Gemma-preview
pip install sentence-transformers>=5.0.0
```

👍
John6666
6317233cc92fd6fee317e030
https://cdn-avatars.huggingface.co/v1/production/uploads/6317233cc92fd6fee317e030/cJHSvvimr1kqgQfHOjO5n.png
Tom Aarsen
tomaarsen
user
645dbaa6f5760d1530d7580d
https://cdn-avatars.huggingface.co/v1/production/uploads/645dbaa6f5760d1530d7580d/Bqob8arLZoHIgMwNZpL9I.jpeg
Simeon Emanuilov
s-emanuilov
user