You are happy that 
text
You are happy that 
@Meta
mention
Meta
 has open-sourced Llama 3 😃...
text
 has open-sourced Llama 3 😃...
new_line
new_line
So you jump on 
text
So you jump on 
@HuggingFace
mention
HuggingFace
 Hub to download the new shiny Llama 3 model only to see a few quintillion Llama 3's! 🦙✨
text
 Hub to download the new shiny Llama 3 model only to see a few quintillion Llama 3's! 🦙✨
new_line
new_line
Which one should you use? 🤔
text
Which one should you use? 🤔
new_line
new_line
Not all Llamas are created equal! 🦙⚖️
text
Not all Llamas are created equal! 🦙⚖️
new_line
new_line
An absolutely crazy comparison experiment by Wolfram Ravenwolf (
text
An absolutely crazy comparison experiment by Wolfram Ravenwolf (
@Wolfram
mention
Wolfram
) might answer your question! 🧪🧙‍♂️
text
) might answer your question! 🧪🧙‍♂️
new_line
new_line
- Comprehensive assessment of Llama 3 Instruct 70B and 8B models. 📊
text
- Comprehensive assessment of Llama 3 Instruct 70B and 8B models. 📊
new_line
- Tested 20 versions across HF, GGUF, and EXL2 formats. 🔄
text
- Tested 20 versions across HF, GGUF, and EXL2 formats. 🔄
new_line
- Methodology: The process tested translation capabilities and cross-language understanding, using deterministic generation settings to minimize random factors. Used German data protection training exams to evaluate cross-language understanding. 🌐📝
text
- Methodology: The process tested translation capabilities and cross-language understanding, using deterministic generation settings to minimize random factors. Used German data protection training exams to evaluate cross-language understanding. 🌐📝
new_line
- Best performance from EXL2 4.5bpw quant, scoring perfect in all tests. 🏆✅
text
- Best performance from EXL2 4.5bpw quant, scoring perfect in all tests. 🏆✅
new_line
- GGUF 8-bit to 4-bit quants also performed exceptionally. 🌟
text
- GGUF 8-bit to 4-bit quants also performed exceptionally. 🌟
new_line
- Llama 3 8B unquantized is best in its size class but not as good as 70B quants. 📏🔍
text
- Llama 3 8B unquantized is best in its size class but not as good as 70B quants. 📏🔍
new_line
- 1-bit quantizations showed significant quality drops. ⚠️⬇️
text
- 1-bit quantizations showed significant quality drops. ⚠️⬇️
new_line
new_line
Best models: 
text
Best models: 
new_line
- 
text
- 
https://huggingface.co/turboderp/Llama-3-70B-Instruct-exl2/tree/5.0bpw
turboderp/Llama-3-70B-Instruct-exl2
model
resource
https://huggingface.co/turboderp/Llama-3-70B-Instruct-exl2/tree/5.0bpw
new_line
- 
text
- 
https://huggingface.co/casperhansen/llama-3-70b-instruct-awq
casperhansen/llama-3-70b-instruct-awq
model
resource
https://huggingface.co/casperhansen/llama-3-70b-instruct-awq
new_line
new_line
Blog: 
text
Blog: 
https://huggingface.co/blog/wolfram/llm-comparison-test-llama-3
https://huggingface.co/blog/wolfram/llm-comparison-test-llama-3
link
You are happy that @Meta has open-sourced Llama 3 😃...

So you jump on @HuggingFace Hub to download the new shiny Llama 3 model only to see a few quintillion Llama 3's! 🦙✨

Which one should you use? 🤔

Not all Llamas are created equal! 🦙⚖️

An absolutely crazy comparison experiment by Wolfram Ravenwolf (@Wolfram) might answer your question! 🧪🧙‍♂️

- Comprehensive assessment of Llama 3 Instruct 70B and 8B models. 📊
- Tested 20 versions across HF, GGUF, and EXL2 formats. 🔄
- Methodology: The process tested translation capabilities and cross-language understanding, using deterministic generation settings to minimize random factors. Used German data protection training exams to evaluate cross-language understanding. 🌐📝
- Best performance from EXL2 4.5bpw quant, scoring perfect in all tests. 🏆✅
- GGUF 8-bit to 4-bit quants also performed exceptionally. 🌟
- Llama 3 8B unquantized is best in its size class but not as good as 70B quants. 📏🔍
- 1-bit quantizations showed significant quality drops. ⚠️⬇️

Best models: 
- https://huggingface.co/turboderp/Llama-3-70B-Instruct-exl2/tree/5.0bpw
- https://huggingface.co/casperhansen/llama-3-70b-instruct-awq

Blog: https://huggingface.co/blog/wolfram/llm-comparison-test-llama-3
🤗
GPT007 clem jensbosseparra not-lain Ramikan-BR
👍
dillfrescott not-lain Ramikan-BR wsuff
🔥
Citaman not-lain Ramikan-BR
👀
Ramikan-BR