Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋
text
Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋
new_line
GGUFs: 
text
GGUFs: 
https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF
unsloth/DeepSeek-V3.1-GGUF
model
resource
https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF
new_line
new_line
The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.
text
The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.
new_line
new_line
The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.
text
The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.
new_line
new_line
Guide: 
text
Guide: 
https://docs.unsloth.ai/basics/deepseek-v3.1
https://docs.unsloth.ai/basics/deepseek-v3.1
link
Run DeepSeek-V3.1 locally on 170GB RAM with Dynamic 1-bit GGUFs!🐋
GGUFs: https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF

The 715GB model gets reduced to 170GB (-80% size) by smartly quantizing layers.

The 1-bit GGUF passes all our code tests & we fixed the chat template for llama.cpp supported backends.

Guide: https://docs.unsloth.ai/basics/deepseek-v3.1
❤️
imweijh shimmyshimmer Rogeriotomas John6666 josancamon GarroshIcecream lofitolstoy yarikdevcom glenn2 Alkohole K-ai-Innovations Koushal1994 sequelbox myahya8623 Wanderspool Orenguteng fausap wilsoncheng WaterGuyV1
🔥
shimmyshimmer josancamon yarikdevcom yanuararif aquiffoo Wanderspool sulfierry Orenguteng fausap
🚀
shimmyshimmer yarikdevcom Wanderspool Orenguteng fausap