Try out Mixtral 2-bit on a free-tier Google Colab notebook right now! 
text
Try out Mixtral 2-bit on a free-tier Google Colab notebook right now! 
new_line
new_line
https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing
https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing
link
text
new_line
new_line
AQLM method has been recently introduced on transformers main branch 
text
AQLM method has been recently introduced on transformers main branch 
new_line
new_line
The 2bit model can be found here: 
text
The 2bit model can be found here: 
https://huggingface.co/BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
model
resource
https://huggingface.co/BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch
text
new_line
new_line
And you can read more about the method here: 
text
And you can read more about the method here: 
https://huggingface.co/docs/transformers/main/en/quantization#aqlm
https://huggingface.co/docs/transformers/main/en/quantization#aqlm
link
new_line
new_line
Great work 
text
Great work 
@BlackSamorez
mention
BlackSamorez
 and team!
text
 and team!
Try out Mixtral 2-bit on a free-tier Google Colab notebook right now! 

https://colab.research.google.com/drive/1-xZmBRXT5Fm3Ghn4Mwa2KRypORXb855X?usp=sharing 

AQLM method has been recently introduced on transformers main branch 

The 2bit model can be found here: https://huggingface.co/BlackSamorez/Mixtral-8x7b-AQLM-2Bit-1x16-hf-test-dispatch 

And you can read more about the method here: https://huggingface.co/docs/transformers/main/en/quantization#aqlm

Great work @BlackSamorez and team!
❤️
osanseviero alielfilali01 sayhan Chunte victor samusenps macadeliccc clem Dlbk andysalerno jeffboudier loubnabnl linoyts medmac01 mathiasn1 Ji-Ha kramp bombassaro
65c5f0a7e03f78fa9efbbd77
/avatars/3fb9c2725e25fc91b6f72a41f632c8d0.svg
Vishvesh Mangal
Kingvishy
user
62441d1d9fdefb55a0b7d12c
https://cdn-avatars.huggingface.co/v1/production/uploads/1648631057413-noauth.png
Younes B
ybelkada
user
5f5ac23710b2753d9000c864
/avatars/a5d3be1c3e9df8a367922d7aa3214cca.svg
Kevin Amiri
kevin009
user
64aa878281dda4817269395d
https://cdn-avatars.huggingface.co/v1/production/uploads/64aa878281dda4817269395d/OkeiTI-2SU0C86DXG3fNT.png
koo
K00B404
user