Harsha001/Drama-3-ROCmFPX-GGUF

Drama 3 (35B-A3B) — ROCmFPX GGUF

Built with Fish Audio.

Release notes

31 October 2026: final release of Drama 3

• All five files were rebuilt from the final Drama 3 weights (fishaudio/drama-3 revision 78e5ab7, released 29 October). They replace the files of 20 October, which came from the preview weights (revision e0a9218). The filenames are unchanged. If you downloaded before 31 October, download again.
• New: ROCmFPX tensor scales (see Tensor scales). The 8-bit file's error rate dropped from 3.69% CER (preview file) to 1.94%, and every file is now within the F16 reference's seed-to-seed spread.
• New speechcpp-rocmfpx.patch for speech.cpp v0.8.10: adds the tensor-scale support and pins the converter to the final release. The old patch does not know the scale tensors, so use the new one. The forced-include shim is no longer needed.
• New recommendation: Q4_0_ROCMFP4 (CER 1.18%, F16 1.44%).

20 October 2026: first upload (Drama 3 preview weights, revision e0a9218).

Fish Audio Drama 3 (35B parameters, 3B active; speech in 83 languages, one speaker or a scene of up to 16, with cues such as [laughs] in the text) quantized to ROCmFPX weight types for speech.cpp. Each file holds the whole model, its audio decoder and its 44.1 kHz codec.

These files run only in speech.cpp built against the ROCmFPX fork of ggml, with the patch in this repo (speechcpp-rocmfpx.patch). llama.cpp, LM Studio, Ollama and stock speech.cpp cannot run them.

Not made, sponsored or endorsed by Fish Audio. Use is subject to the Fish Audio Community License 1.0 (LICENSE, NOTICE): free for research and non-commercial use, commercial use only below USD 500,000 yearly revenue, and no hosted speech or voice-cloning service without a separate agreement. Do not imitate a real person's voice without their consent.

Files and measured quality

Each file spoke the same 20 English sentences (about 110 s of speech) with --voice none at seeds 1 and 2. Qwen3-ASR 1.7B transcribed the audio, and CER / WER are the character and word error rates of that transcript against the text (lower is better; mean of the two seeds). The reference is the F16 file (the F32 file does not fit in the test machine's memory). The same file moves by about a point from seed to seed, so differences of a point or less between files are noise.

File	Size	CER	WER	CER per seed	Speed on the test CPU
F16 (reference, not published)	71.6 GB	1.44%	1.66%	1.90 / 0.98	0.10x realtime
Drama-35B-A3B-v3-Q8_0_ROCMFPX.gguf	37.4 GB	1.94%	1.85%	2.89 / 0.98	0.39x realtime
Drama-35B-A3B-v3-Q6_K_ROCMFPX.gguf	29.8 GB	1.31%	1.48%	1.64 / 0.98	0.02x realtime
Drama-35B-A3B-v3-Q4_0_ROCMFP4.gguf	21.1 GB	1.18%	1.29%	0.98 / 1.38	0.20x realtime
Drama-35B-A3B-v3-Q4_0_ROCMFP4_FAST.gguf	20.1 GB	1.84%	2.03%	1.71 / 1.97	0.10x realtime
Drama-35B-A3B-v3-Q4_0_ROCMI4.gguf	20.1 GB	1.50%	1.66%	0.98 / 2.03	0.10x realtime
Q4_K (stock ggml, baseline only)	20.8 GB	2.26%	2.58%	3.54 / 0.98	0.49x realtime

All published files are within noise of F16. Q4_0_ROCMFP4 is the recommended file for AMD GPUs.

The 6-bit file is named Drama-35B-A3B-v3-Q6_K_ROCMFPX.gguf so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock llama.cpp Q6_K, and it needs the same fork as the other files.

Speed is seconds of speech per second of compute on the 20-core ARM CPU of the test machine (NVIDIA DGX Spark), where the ROCmFPX types run on their portable CPU code. The 6-bit type has no fast ARM path, so it is very slow there; on AMD GPUs the types run on dedicated kernels. These files have not been tested on AMD hardware yet; speed reports from AMD GPUs are welcome.

Tensor scales

ROCmFPX block scales are small floating-point numbers with a lower limit. Most of Drama 3's expert weights are so small that their blocks hit that limit, which cost the 8-bit type about 4 bits of precision in the October 20 files. The new files multiply each weight matrix (each expert of a mixture-of-experts matrix) by a power of two before it is quantized and store the inverse as a small <name>.scale tensor, which speech.cpp multiplies back after the matrix product. This is the same scheme as LLAMA_ROCMFPX_TENSOR_SCALE=1 in the ROCmFPX fork of llama.cpp. The lookup tables that are read row by row (token and codebook embeddings) are stored as stock Q8_0 instead.

How these were made

1. reference/drama3/convert.py from speech.cpp (v0.8.10), pinned to the final release (revision 78e5ab7, FP8 experts dequantized as fish-speech does), gives one F32 GGUF of 143 GB.
2. speech.cpp v0.8.10 built on the ROCmFPX ggml with speechcpp-rocmfpx.patch: the ROCmFPX types in its storage tables and the Drama 3 layout, the tensor-scale support (quantizer, loader and matrix products), and the converter pinned to the final release.
3. LLAMA_ROCMFPX_TENSOR_SCALE=1 speech quantize Drama-35B-A3B-v3-F32.gguf OUT --type <type>. The layout decides each tensor's type: the matrices of the text model and the audio decoder take the type, the codec stays F16, and norms, the router and other sensitive tensors stay F32. No importance matrix (speech.cpp does not support one).
4. Speech generated on the CPU (ARM, NVIDIA DGX Spark) with the ROCmFPX CPU kernels.

Run

git clone https://github.com/HarshaVardhan3002/ROCmFPX
git clone https://github.com/nyosegawa/speech.cpp && cd speech.cpp && git checkout v0.8.10
git apply ../speechcpp-rocmfpx.patch
rm -rf ggml && ln -s ../ROCmFPX/ggml ggml          # build on the ROCmFPX ggml instead of the submodule
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_VULKAN=ON
cmake --build build -j
build/speech tts Drama-35B-A3B-v3-Q4_0_ROCMFP4.gguf --voice none -o out.wav \
  "<|speaker:0|> Did you hear the news? <|speaker:1|> [laughs] Yes, I still can't believe it."

Voices, multi-speaker scenes and every option: speech.cpp's Drama 3 guide.