Instructions to use RadixArk/Muse-Glimmer-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use RadixArk/Muse-Glimmer-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="RadixArk/Muse-Glimmer-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("RadixArk/Muse-Glimmer-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("RadixArk/Muse-Glimmer-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use RadixArk/Muse-Glimmer-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "RadixArk/Muse-Glimmer-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Muse-Glimmer-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/RadixArk/Muse-Glimmer-NVFP4

SGLang

How to use RadixArk/Muse-Glimmer-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "RadixArk/Muse-Glimmer-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Muse-Glimmer-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "RadixArk/Muse-Glimmer-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "RadixArk/Muse-Glimmer-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use RadixArk/Muse-Glimmer-NVFP4 with Docker Model Runner:
```
docker model run hf.co/RadixArk/Muse-Glimmer-NVFP4
```

Sync chat template with meta-models/Muse-Glimmer-30B (fixes duplicate reasoning effort)

by Jiminator - opened 21 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

+211

-1

Jiminator

RadixArk org 21 days ago

This repo ships a stale copy of the Muse Glimmer chat template (7,175 bytes,
md5 6dba2759). Upstream fixed it in
meta-models/Muse-Glimmer-30B#35;
the current template is 9,992 bytes, md5 216c1d8e. This copies it verbatim.

Why this matters

The stale template appends its own reasoning directive unconditionally. If the
caller's system prompt already specifies one, the model receives two contradictory
instructions and the appended one wins:

system prompt	stale template renders	current template renders
`Reasoning effort: low.`	`Reasoning effort: low.` + `Reasoning strength: high.`	`Reasoning strength: low.`
`Reasoning strength: low.`	`Reasoning strength: low.` + `Reasoning strength: high.`	`Reasoning strength: low.`
(none)	`Reasoning strength: high.`	identical

The current template normalises "Reasoning effort" to "Reasoning strength" across
casings and only appends when the system prompt has no directive of its own.

Measured impact

Same prompt, same weights, greedy, on Muse-Glimmer-q4km-gs128-MLX via mlx-lm on
an M5 Pro, system prompt asking for low reasoning effort:

	stale template	current template
reasoning characters	160	102
total generated	225	167
wall clock	7.5 s	3.0 s
answer	correct	correct

So a user asking for low effort gets 57% more reasoning and 2.5x the latency for
an identical answer. It only manifests when the system prompt mentions reasoning
effort, which is why a default render is byte-identical and this went unnoticed.

Verification

No inference-code change is needed for the new template: serving
meta-models/Muse-Glimmer-30B (which already carries it) on stock upstream SGLang
passes the full tool-call suite 5/5 and splits reasoning correctly. The delta
between the two templates is mostly a minified-to-pretty-printed reformat plus the
one conditional above; none of the channel markers SGLang parses changed.

Sync chat template with meta-models/Muse-Glimmer-30B (fixes duplicate reasoning effort)464ea5d7

Jiminator

RadixArk org 21 days ago

NVFP4-specific verification (added after opening)

The measurement in the description above was taken on MLX q4km-gs128. Re-ran the
identical comparison on this checkpoint — RadixArk/Muse-Glimmer-NVFP4 served
on a single B300 via stock upstream SGLang, greedy, same prompt asking for low
reasoning effort:

	stale template (shipped here)	current template
directives in prompt	`Reasoning effort: low.` + `Reasoning strength: high.`	`Reasoning strength: low.`
total generated chars	235	130

45% more generation for an identical request, matching the 36% seen on MLX. Both
quantizations show the same effect, so this is a property of the template rather
than of any one artifact.

Caveat on methodology: this run drove /generate with raw rendered prompts, so the
reasoning/answer split parsing does not apply as cleanly as in the MLX harness — the
reliable metric here is total generated length. The MLX figures in the description
remain the cleaner measurement.

Jiminator changed pull request status to merged 21 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment