LeonaQuantum/leonaq-1.5-mini-mac-optimized

Leona-Q 1.5 Mini — Mac Optimized

A Mac-optimized inference runtime for LeonaQuantum/leonaq-1.5-mini. This is llama.cpp + Metal, not MLX, and not a newly trained or additionally quantized model. The official Q4_K_M base weights, learned butterfly checkpoint, all 42 transformer layers and both circuit layers are retained.

The model uses classical statevector simulation of learned quantum circuits; no quantum hardware is required. Use the included runtime: stock llama.cpp, LM Studio and Transformers do not execute this adapter as-is.

Quick start on Apple Silicon

The bundled ARM64 executables require macOS 15 or later and were built and tested on M1 / 7 GPU cores / 8 GB RAM / macOS 15.5. Compatibility and performance on other Macs have not been measured. Source builds are available below.

With Python 3.12+ and the Hugging Face CLI installed:

hf download LeonaQuantum/leonaq-1.5-mini-mac-optimized --local-dir leonaq-1.5-mini-mac-optimized
cd leonaq-1.5-mini-mac-optimized
chmod +x bin/llama-* baseline/bin/llama-* v1/bin/llama-*
python3 fetch.py
python3 mac.py --server

fetch.py downloads the unchanged 1.56 GB base from its original publisher and verifies its pinned SHA-256. The base GGUF is not duplicated in this repository. Open http://127.0.0.1:8080 for the included chat UI. Stop with Ctrl-C. Inference runs locally.

To explicitly try the optional verified acceleration mode:

python3 mac.py --server --acceleration verified --threads 2

Every proposed token is checked with the full target model and circuit. Small batches retain the existing F32 matrix-vector reductions; no auxiliary draft model or saved-answer corpus is used. This mode can be slower on some inputs, so it is optional.

For a single CLI request:

python3 mac.py --prompt '日本語で自己紹介してください。' --max-tokens 512

The token limit includes reasoning and can truncate an answer. The launcher does not disable reasoning to improve apparent speed.

What improved, and what was measured

• Portable C++ circuit optimizations: separate real/imaginary arrays for compiler SIMD, prepared fixed CNOT permutations, reused initial states and amplitude norms. Learned gates, routing and coherence remain active.
• Selective parallel Metal initialization, with complete on-demand fallback for operators not prepared at startup.
• Optional verified speculative generation with GPU column layout changes to reuse weights, preserving the original F32 reductions.
• Per-Mac measurement of thread counts and optional acceleration, with exact-output and regression gates.
• RAM-based physical microbatch sizing, retaining context and numerical precision.

In a small local comparison of four tasks, verified mode achieved 1.136x aggregate throughput relative to the standard configuration. Some inputs were slower, and repeated timings varied substantially. This is not a universal speed claim. It compares two generation threads in verified mode with four in standard mode, and includes input processing. The speed trial used physical microbatch 512; the current 8 GB default is 128 after a later memory-pressure test.

All 2,658 tokens across the two verified repetitions matched the standard reference. A separate eight-topic check matched 1,024 tokens, and a temperature-0.7 check matched 256 tokens. Five GPU variants matched 86,032 output floats each bit for bit; the batch kernels matched 16,384 values. Six developer test suites passed. These finite checks do not establish full task accuracy or universal cross-device bitwise identity. The earlier CPU circuit rewrite allows small floating-point rounding differences, checked with absolute tolerance 2e-5.

See V2_RESULTS.md for conditions, per-task results, unsuccessful trials and limitations. MAC_RESULTS.md preserves the earlier V1 results. Upstream benchmark scores in EVALUATION.md are not new Mac accuracy measurements.

Tune for a particular Mac

python3 mac.py --tune --tune-tokens 512
python3 mac.py --server

Calibration runs uncached generations and saves local prompts, outputs and timings. A candidate must match output tokens and stop conditions, improve aggregate speed by at least 5%, and avoid any measured case becoming more than 10% slower. Otherwise the standard configuration remains selected. Run calibration without competing inference workloads. No preselected performance profile is shipped.

Profiles bind to CPU/GPU identity, RAM, macOS, runtime hashes, backend environment, model and circuit. Incompatible profiles are ignored. Manual --threads, --threads-batch and --acceleration settings override calibration. --no-profile --acceleration off selects standard inference.

Defaults are 4096 context tokens, one server slot, F16 KV, logical batch 512, and physical microbatch 128/256/512 for <=8 GB / <=16 GB / larger Macs. --context, --threads, --threads-batch, --batch, --ubatch and --backend cpu override them. Physical microbatch limits temporary memory without reducing context or precision. Other applications can still exhaust available memory.

Build on another Mac

Install Python 3.12+, CMake and Xcode Command Line Tools, then:

python3 build.py --backend metal --portable --verified-speculation --jobs 3
python3 fetch.py
python3 mac.py --tune --tune-tokens 512
python3 mac.py --server

build.py downloads the pinned llama.cpp source and applies the included patches. mac.py prefers a local build-metal/bin over the packaged binaries. --portable disables host-specific CPU tuning; it does not create an Intel/ARM universal binary. There is no M1-only model-name branch.

Intel Macs can build python3 build.py --backend cpu --portable, then use python3 mac.py --backend cpu --bin-dir build-cpu/bin --server. The ARM64 binaries and verified Metal acceleration do not support Intel. Intel and other M-series hardware have not been tested here.

Windows has source build targets (--backend cuda, vulkan or cpu) through the included portable runtime, but these Mac binaries and Metal-specific optimizations do not run on Windows. Full-model Windows compatibility and performance remain unverified.

Reproduce and inspect

Source patches, developer tests, previous runtimes (v1/bin, baseline/bin) and raw measurement logs are included. Binary library aliases are materialized as ordinary files for Hub downloads. OpenSSL 3.6.3 libraries are included under runtime-deps/ with their license; packaged dependency paths are relative, so Homebrew is not required for these binaries. Packaging changes dependency paths and ad-hoc signatures, not inference arithmetic. Raw benchmark hashes identify the binaries before this packaging step; RELEASE.json records the affected files and their before/after hashes. SHA256SUMS.json covers the published files. Recorded local usernames and home/workspace paths are replaced with placeholders; prompts, outputs and timings are unchanged.

python3 experiments_v2.py --bin-dir bin --model MiniCPM5-2B-Q4_K_M.gguf --output checks/local-comparison --modes baseline,verified --tokens 512 --ubatch 128

After a source build, run the developer suites:

cmake -S tests -B build-checks -DCMAKE_BUILD_TYPE=Release -DLEONA_GGML_LIBRARY_DIR="$PWD/build-metal/bin"
cmake --build build-checks --parallel 2
ctest --test-dir build-checks --output-on-failure

The base, circuit checkpoint and runtime retain their original licenses. See ATTRIBUTION.md, LICENSE, licenses/llama.cpp-MIT.txt and the preserved upstream model card. Source revisions and checksums are recorded in butterfly_manifest.json and bin/leona-build.json.