I'm now working on finetuning of coding models. If you are GPU-hungry like me, you will find quantized models very helpful. But quantization for finetuning and inference are different and incompatible. So I made two collections here.
text
I'm now working on finetuning of coding models. If you are GPU-hungry like me, you will find quantized models very helpful. But quantization for finetuning and inference are different and incompatible. So I made two collections here.
new_line
new_line
Inference (GGUF, via Ollama, CPU is enough)
text
Inference (GGUF, via Ollama, CPU is enough)
new_line
https://huggingface.co/collections/onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6
onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6
collection
resource
https://huggingface.co/collections/onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6
new_line
new_line
Finetuning (Bitsandbytes, QLora, GPU is needed)
text
Finetuning (Bitsandbytes, QLora, GPU is needed)
new_line
https://huggingface.co/collections/onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2
onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2
collection
resource
https://huggingface.co/collections/onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2
new_line
new_line
For quantization, the inference models are far more popular on HF than finetuning models. I use 
text
For quantization, the inference models are far more popular on HF than finetuning models. I use 
https://cdn-avatars.huggingface.co/v1/production/uploads/6382255fcae34727b9cc149e/ANRA_7hWosC6_2PS2cwtg.jpeg
https://huggingface.co/QuantFactory
QuantFactory
org
resource
https://huggingface.co/QuantFactory
 to generate inference models (GGUF), and there are a few other choices.
text
 to generate inference models (GGUF), and there are a few other choices.
new_line
new_line
But there hasn't been such a service for finetuning models. DIY isn't too hard though. I made a few myself and you can find the script in the model cards. If the original model is small enough, you can even do it on a free T4 (available via Google Colab).
text
But there hasn't been such a service for finetuning models. DIY isn't too hard though. I made a few myself and you can find the script in the model cards. If the original model is small enough, you can even do it on a free T4 (available via Google Colab).
new_line
new_line
If you know a (small) coding model worthy of quantization, please let me know and I'd love to add it to the collections.
text
If you know a (small) coding model worthy of quantization, please let me know and I'd love to add it to the collections.
I'm now working on finetuning of coding models. If you are GPU-hungry like me, you will find quantized models very helpful. But quantization for finetuning and inference are different and incompatible. So I made two collections here.

Inference (GGUF, via Ollama, CPU is enough)
https://huggingface.co/collections/onekq-ai/ollama-ready-coding-models-67118c3cfa1af2cf04a926d6

Finetuning (Bitsandbytes, QLora, GPU is needed)
https://huggingface.co/collections/onekq-ai/qlora-ready-coding-models-67118771ce001b8f4cf946b2

For quantization, the inference models are far more popular on HF than finetuning models. I use https://huggingface.co/QuantFactory to generate inference models (GGUF), and there are a few other choices.

But there hasn't been such a service for finetuning models. DIY isn't too hard though. I made a few myself and you can find the script in the model cards. If the original model is small enough, you can even do it on a free T4 (available via Google Colab).

If you know a (small) coding model worthy of quantization, please let me know and I'd love to add it to the collections.
🔥
djuna John6666 valentimarco RalFinger rajkumar321 regisss