When 
text
When 
@MistralAI
mention
MistralAI
 drops a blog post labelled "Large Enough," it's going to get serious! 🚀💡
text
 drops a blog post labelled "Large Enough," it's going to get serious! 🚀💡
new_line
new_line
- Mistral-Large-Instruct-2407, just call it Mistral-Large2, is a 123B parameters Instruct model with 128k context 🌍📚
text
- Mistral-Large-Instruct-2407, just call it Mistral-Large2, is a 123B parameters Instruct model with 128k context 🌍📚
new_line
new_line
- Multilingual in 11 languages; English 🇬🇧, French 🇫🇷, German 🇩🇪, Spanish 🇪🇸, Italian 🇮🇹, Chinese 🇨🇳, Japanese 🇯🇵, Korean 🇰🇷, Portuguese 🇵🇹, Dutch 🇳🇱, and Polish 🇵🇱. 🗣️🗺️
text
- Multilingual in 11 languages; English 🇬🇧, French 🇫🇷, German 🇩🇪, Spanish 🇪🇸, Italian 🇮🇹, Chinese 🇨🇳, Japanese 🇯🇵, Korean 🇰🇷, Portuguese 🇵🇹, Dutch 🇳🇱, and Polish 🇵🇱. 🗣️🗺️
new_line
new_line
- Also highly focused on programming, trained on 80+ coding languages such as Python, Java, C, C++, Javascript, bash 💻🔧
text
- Also highly focused on programming, trained on 80+ coding languages such as Python, Java, C, C++, Javascript, bash 💻🔧
new_line
new_line
- Supports native function calling and structured output. 🛠️📊
text
- Supports native function calling and structured output. 🛠️📊
new_line
new_line
- Released under Mistral Research License (Non-Commercial License, Research only😔) 
text
- Released under Mistral Research License (Non-Commercial License, Research only😔) 
new_line
new_line
- Open weights only🔓, no data or code released 🔒📁
text
- Open weights only🔓, no data or code released 🔒📁
new_line
new_line
Definitely firing shots at 
text
Definitely firing shots at 
@Meta
mention
Meta
 Llama3.1: 🎯🔥
text
 Llama3.1: 🎯🔥
new_line
MMLU - 84.0% (ML2) vs 79.3% (L3.1-70B) vs 85.2% (L3.1-405B)
text
MMLU - 84.0% (ML2) vs 79.3% (L3.1-70B) vs 85.2% (L3.1-405B)
new_line
GSM8K - 93% (ML2) vs 95.5% (L3.1-70B-Ins) vs 96.8% (L3.1-405B-Ins)
text
GSM8K - 93% (ML2) vs 95.5% (L3.1-70B-Ins) vs 96.8% (L3.1-405B-Ins)
new_line
new_line
Also, it's kinda chunky! 📦💪
text
Also, it's kinda chunky! 📦💪
new_line
fp16/ bf16 - ~250GB VRAM
text
fp16/ bf16 - ~250GB VRAM
new_line
fp8/ int8 - ~125GB VRAM
text
fp8/ int8 - ~125GB VRAM
new_line
int4 - ~60GB VRAM
text
int4 - ~60GB VRAM
new_line
new_line
I tried quantising it to AWQ and GPTQ, but couldn't with 30GB V-RAM. ❌🖥️
text
I tried quantising it to AWQ and GPTQ, but couldn't with 30GB V-RAM. ❌🖥️
new_line
new_line
Also calling out AWQ and GPTQ on not supporting multi-GPU quantisation! 🖥️⚡
text
Also calling out AWQ and GPTQ on not supporting multi-GPU quantisation! 🖥️⚡
new_line
new_line
God sent 
text
God sent 
@casperhansen
mention
casperhansen
 has posted AWQ quantised INT4 model (68.68 GB) with the perplexity of 2.889: 
text
 has posted AWQ quantised INT4 model (68.68 GB) with the perplexity of 2.889: 
https://huggingface.co/casperhansen/mistral-large-instruct-2407-awq
casperhansen/mistral-large-instruct-2407-awq
model
resource
https://huggingface.co/casperhansen/mistral-large-instruct-2407-awq
 🔥👏
text
 🔥👏
new_line
new_line
Looks like open AI is going to beat OpenAI! 🏆🤖
text
Looks like open AI is going to beat OpenAI! 🏆🤖
new_line
new_line
Blog post: 
text
Blog post: 
https://mistral.ai/news/mistral-large-2407/
https://mistral.ai/news/mistral-large-2407/
link
new_line
new_line
Models: 
text
Models: 
https://huggingface.co/mistralai/Mistral-Large-Instruct-2407
mistralai/Mistral-Large-Instruct-2407
model
resource
https://huggingface.co/mistralai/Mistral-Large-Instruct-2407
When @MistralAI drops a blog post labelled "Large Enough," it's going to get serious! 🚀💡

- Mistral-Large-Instruct-2407, just call it Mistral-Large2, is a 123B parameters Instruct model with 128k context 🌍📚

- Multilingual in 11 languages; English 🇬🇧, French 🇫🇷, German 🇩🇪, Spanish 🇪🇸, Italian 🇮🇹, Chinese 🇨🇳, Japanese 🇯🇵, Korean 🇰🇷, Portuguese 🇵🇹, Dutch 🇳🇱, and Polish 🇵🇱. 🗣️🗺️

- Also highly focused on programming, trained on 80+ coding languages such as Python, Java, C, C++, Javascript, bash 💻🔧

- Supports native function calling and structured output. 🛠️📊

- Released under Mistral Research License (Non-Commercial License, Research only😔) 

- Open weights only🔓, no data or code released 🔒📁

Definitely firing shots at @Meta Llama3.1: 🎯🔥
MMLU - 84.0% (ML2) vs 79.3% (L3.1-70B) vs 85.2% (L3.1-405B)
GSM8K - 93% (ML2) vs 95.5% (L3.1-70B-Ins) vs 96.8% (L3.1-405B-Ins)

Also, it's kinda chunky! 📦💪
fp16/ bf16 - ~250GB VRAM
fp8/ int8 - ~125GB VRAM
int4 - ~60GB VRAM

I tried quantising it to AWQ and GPTQ, but couldn't with 30GB V-RAM. ❌🖥️

Also calling out AWQ and GPTQ on not supporting multi-GPU quantisation! 🖥️⚡

God sent @casperhansen has posted AWQ quantised INT4 model (68.68 GB) with the perplexity of 2.889: https://huggingface.co/casperhansen/mistral-large-instruct-2407-awq 🔥👏

Looks like open AI is going to beat OpenAI! 🏆🤖

Blog post: https://mistral.ai/news/mistral-large-2407/

Models: https://huggingface.co/mistralai/Mistral-Large-Instruct-2407