eric717273/Qwen3.5-122B-A10B-MXFP4_MOE-MTP-GGUF

Qwen3.5-122B-A10B MXFP4_MOE GGUF with MTP Donor

This is a community-merged GGUF build of Qwen3.5-122B-A10B MXFP4_MOE with an injected MTP donor layer for llama.cpp MTP speculative decoding.

This repository is not an official release from Qwen, Unsloth, or LaikaFramework.

This is an experimental community merge. Please report loading issues, MTP compatibility problems, or runtime results in the Discussions tab.

What this repository contains

This repository provides a merged GGUF file:

Qwen3.5-122B-A10B-MXFP4_MOE-MTP-merged.gguf

The model is intended for testing with a custom llama.cpp build that supports MTP speculative decoding.

Sources and attribution

This file was produced from the following sources:

• Original base model: Qwen/Qwen3.5-122B-A10B
• Base GGUF quantization: unsloth/Qwen3.5-122B-A10B-GGUF, MXFP4_MOE
• MTP donor: LaikaFramework/Qwen3.5-122B-A3B-MTP-Donor
• Merge method: convert.py merge <single-base.gguf> <donor.gguf> <output.gguf>
• Runtime: custom llama.cpp build with MTP support, PR #22673 style

Why this merge exists

At the time of testing, complete Qwen3.5-122B-A10B GGUF files with working MTP support were not widely available.
This merge adds the missing MTP/NextN layer metadata and tensors to an existing MXFP4_MOE GGUF base.

The key metadata expected after merge is:

general.architecture = qwen35moe
qwen35moe.block_count = 49
qwen35moe.nextn_predict_layers = 1

Tested environment

Local test environment:

OS: Ubuntu 24.04
Hardware: AMD Ryzen AI Max+ 395 / Radeon 8060S
Memory: 128GB unified memory
Backend: llama.cpp Vulkan / RADV
Context tested: 262144
MTP setting: --spec-type mtp --spec-draft-n-max 2

Local test results

On the above AMD/Vulkan system, the model successfully loaded as:

qwen35moe_mtp

Observed local performance:

1200-token test:
  decode speed: around 31 tok/s
  MTP draft acceptance: around 72%

3000-token test:
  decode speed: around 31 tok/s
  MTP draft acceptance: around 76%

These numbers are only local observations. Your performance may vary significantly depending on backend, hardware, memory bandwidth, context size, and llama.cpp version.

Recommended llama-server command

Example command:

LLAMA="/path/to/llama-server"
MODEL="/path/to/Qwen3.5-122B-A10B-MXFP4_MOE-MTP-merged.gguf"

"$LLAMA" \
  -m "$MODEL" \
  --host 0.0.0.0 \
  --port 1236 \
  --alias qwen35-122b-mxfp4-mtp \
  -ngl 999 \
  -c 262144 \
  -np 1 \
  -b 1024 \
  -ub 256 \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --jinja \
  --reasoning off \
  --reasoning-format deepseek \
  --temp 0.3 \
  --top-p 0.95 \
  --top-k 20 \
  --spec-type mtp \
  --spec-draft-n-max 2

Low-memory KV option

For lower KV cache memory usage at very long context, you may test:

--cache-type-k q4_0 \
--cache-type-v q4_0

In local testing, q4 KV reduced memory pressure and remained usable, but q8 KV is recommended as the default quality-first configuration.

Important notes

• This model requires a llama.cpp build with MTP support.
• This model is not guaranteed to work with stock llama.cpp releases that do not support MTP.
• This file is a community merge and should be treated as experimental.
• Do not use this repository as evidence of official Qwen, Unsloth, or LaikaFramework support.
• MTP speculative decoding primarily improves decode/generation speed, not prefill speed.
• Very long context still requires substantial KV cache memory.

Verification

A SHA256 checksum is included in SHA256SUMS.

To verify after download:

sha256sum -c SHA256SUMS

License

The upstream Qwen model is released under Apache-2.0.
This repository follows Apache-2.0 and retains attribution to the upstream projects.

Please also review and respect the license terms of the upstream model and source repositories.