smarter quant
#1
by
bobig
- opened
Bigger quant, noticably smarter. Worth it if you have the space.
10% slower TPS (almost imperceptible at 200 TPS on Mac M4) the extra smarts and instruction following are worth it!
Fantastic model for real-time tasks with humans.
Also check out the latest Agent, it's very human-like, although coding--dunno, seems opinionated :)
https://huggingface.co/nightmedia/Qwen3-4B-Agent-F32-dwq4-mlx
That's an abliterated/heretic version