YAML Metadata Warning:empty or missing yaml metadata in repo card

Check out the documentation for more information.


license: mit base_model: - deepgrove/maple-preview tags: - maple - gguf - dpo - preference-optimization - on-policy-distillation - moe - ternary - apple-silicon library_name: gguf

Maple Pro β€” preference-optimized Maple (20B-A1B ternary MoE)

A single-file GGUF of Maple (deepgrove/maple-preview, MIT) after a full distillation + DPO campaign: the strongest measured variant of this model family, improving or matching the baseline on all 12 evaluated axes.

Results (measured on a fixed local suite, identical seeds/conditions)

Axis Base Maple Maple Pro
Coding probe (26 tasks) 20 24
GSM8K (pooled n=200) 90/100 90.5/100
HumanEval+ (50) 44 44
IFEval official checker (100) 63 65
Tools eval (12) 11 12/12
Knowledge (25) 22 23
Browser-agent probe (10) 3 5
IF-10 / IF-30 7 / 19 8 / 19
DROP-30 20 20
Median tokens/answer 759 570 (βˆ’25%)

First and only configuration (of 17 tested) achieving simultaneous dominance across every axis. All numbers reproducible; eval suite and logs described in the model card below.

Files β€” which one should I use?

File Size Format Use it on
maple-pro.gguf 6.2 GB ternary TQ2_0 + F16 attn Apple Silicon (LM Studio / Metal) β€” fastest and smallest; verified end-to-end. On Windows/Linux see troubleshooting: lower GPU layers / context, or run CPU-only
maple-pro-Q2_K.gguf 7.0 GB standard quants Universal fallback (any GPU/backend on any OS) if ternary load settings can't be tuned

All three carry identical weights (same training), identical chat template and 256K context metadata. Outputs were validated side-by-side at temperature 0: all three answer the same probe set identically.

Windows / Linux troubleshooting (empty responses)

Symptom: model loads in LM Studio on Windows/Linux, but chat replies come back empty.

Cause: the ternary TQ2_0 tensor type has no stable GPU kernels outside Apple's Metal runtime. The maple architecture landed in mainline llama.cpp (v0.4.1) as CPU-only; on CUDA/Vulkan the ternary matmuls are experimental and can produce garbage logits, which makes the model emit EOS immediately β€” an empty reply. This is a runtime/kernel issue, not a model issue: the same GGUF file generates correctly on CPU and on Apple Silicon.

Fixes (in order):

  1. Use maple-pro-Q4_K_M.gguf or maple-pro-Q2_K.gguf β€” standard quant kernels, battle-tested on every backend. Nothing else to configure.
  2. If you must run the ternary original: set GPU offload to 0 (CPU-only) in the load settings. CPU generation is correct.
  3. Keep the llama.cpp runtime up to date in LM Studio (Ctrl+Shift+R β†’ Runtimes); arch support exists since runtime 2.55.
  4. On GPUs with < 16 GB VRAM, lower the context length (the GGUF advertises 256K; the KV cache at full 256K needs ~3.3 GB on top of the weights).

What's inside

  • Full LoRA fusion: 64 attention LoRA tensors and 121 DPO-updated base tensors (norms, expert routers, QK-norms) baked into a single GGUF. Attention projections stored as F16 (mixed precision; layers 0–7 keep the base ternary TQ2_0 attention); ternary expert weights untouched (byte-identical to the official base).
  • Runtime: 6.6 GB, ~1B active parameters.
  • Context: 128K native, GGUF metadata allows 256K β€” verified to work natively with no flags (needle-tested β€” see Context section). No other changes to the model: the 256K build differs from the 128K original by a single metadata field.

Context (measured)

5-needle recall (wikitext-2 haystack, depths 10/30/50/70/90%), same seeds:

Context Mode Recall
16K–131K native 3/5
256K native (no flags) 4/5
256K YaRN Γ—2 4/5 (same)
384K YaRN Γ—3 2/5 β€” fails

No length degradation up to 256K: the hybrid SWA design (18/24 layers at window 512) extrapolates Γ—2 natively, so 256K works by just setting the context size β€” no rope-scaling flags needed (LM Studio: set the slider). Remaining misses are section-binding confusions between same-shape needles, uniform across lengths. 384K+ is out of reach without long-context training.

Runtime (important)

The maple architecture is not in upstream llama.cpp releases before v0.4.1 (merged CPU-only). Runtimes β‰₯ 2.55 in LM Studio load it; for other builds use the DeepGrove fork (Apache-2.0):

git clone --depth 1 https://github.com/deepgrove-ai/llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --target llama-server

The Metal backend of the current fork does not implement the ternary TQ2_0 matmuls (experts, and the ternary attention of layers 0–7), so serve with those tensors on CPU:

./build/bin/llama-server -m maple-pro.gguf --port 8080 -t 8 \
  --cpu-moe --no-op-offload \
  -ot 'blk\.[0-7]\.attn_[a-z]+\.weight=CPU'

OpenAI-compatible API on :8080. LM Studio works too: the GGUF was verified on LM Studio's bundled runtime (2.65.1) β€” just load it and set the context size. Measured decode on an M4 16 GB (idle): ~75 tok/s (base maple-preview: ~94 tok/s β€” Pro trades some bandwidth for its F16 fused attention).

Serving at 256K

The GGUF advertises a 262144 context: just set the context size to 256K β€” no rope-scaling flags required (native Γ—2 extrapolation is needle-verified, see Context section). On a 16 GB machine use quantized KV cache to fit:

./build/bin/llama-server -m maple-pro.gguf --port 8080 -t 8 \
  --ctx-size 262144 \
  --cpu-moe --no-op-offload \
  -ot 'blk\.[0-7]\.attn_[a-z]+\.weight=CPU' \
  --cache-type-k q8_0 --cache-type-v q8_0

KV cache quantization is your choice; q8_0 was used in the tests (~10 GB total on a 16 GB machine).

Training recipe (summary)

  1. Semi on-policy distillation: the student samples its own solutions on verified tasks (GSM8K gold answers, IFEval-style constraint checkers, sandboxed coding tests); a frontier teacher API model (GLM-5.3-Flash) corrects only the failures. ~3,000 verified on-policy samples + corrections.
  2. SFT on-policy multi-domain (the 11/12-axes intermediate).
  3. DPO on preference pairs (student error ↔ verified teacher correction), full-parameter (attention LoRA + norms/routers), SFT-anchored.
  4. Zero-tolerance gate: a candidate ships only if it matches or beats the baseline on all axes at once. 16 configurations were rejected by this gate before this one passed.

Credits & license

Downloads last month
832
GGUF
Model size
20B params
Architecture
maple
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support