vLLM MoE Config

#32
by j4ys0n - opened

Generated the MoE config that vLLM needs for RTX PRO 6000 Blackwell. Also included an updated script that can be used to produce more configs. Big performance improvement.
https://github.com/MissionSquad/vllm-moe-configs
https://www.reddit.com/r/LocalLLaMA/comments/1oi16jj/vllm_moe_benchmark_configs_for_qwen3_coder_reap/

Sign up or log in to comment