You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs: 
text
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs: 
https://huggingface.co/unsloth/Kimi-K2-Thinking-GGUF
unsloth/Kimi-K2-Thinking-GGUF
model
resource
https://huggingface.co/unsloth/Kimi-K2-Thinking-GGUF
new_line
new_line
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
text
We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.
new_line
new_line
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
text
We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰
new_line
new_line
Guide + fix details: 
text
Guide + fix details: 
https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
link
You can now run Kimi K2 Thinking locally with our Dynamic 1-bit GGUFs: https://huggingface.co/unsloth/Kimi-K2-Thinking-GGUF

We shrank the 1T model to 245GB (-62%) & retained ~85% of accuracy on Aider Polyglot. Run on >247GB RAM for fast inference.

We also collaborated with the Moonshot AI Kimi team on a system prompt fix! 🥰

Guide + fix details: https://docs.unsloth.ai/models/kimi-k2-thinking-how-to-run-locally
❤️
shimmyshimmer Erland mike-ravkine gruber nobodyPerfecZ John6666 sachin6624 unmodeled-tyler zati WaterGuyV1
🚀
shimmyshimmer Erland gruber Parkash15 sachin6624 unmodeled-tyler zati abskvrm WaterGuyV1
🔥
shimmyshimmer Erland gruber DeathGodlike sachin6624 zati
🤗
shimmyshimmer Erland sachin6624 zati
🤯
johnlockejrr sachin6624 zati