Model Compression with Exact Budget Constraints via Riemannian Manifolds
Paper • 2605.00649 • Published • 15
Paretrix Quantization Suite (v3.1.1)
Paretrix is an empirical, activation-aware quantization suite for GGUF language models and speculative decoding modules.
Pareto + Matrix → Paretrix. Each rung is planned so that no better trade exists at its weight among the options measured. Around three stock bases (Q8_0, Q6_K-imx, IQ4_XS-imx), every tensor class is moved alone to every type and measured against the BF16 logits; an exact knapsack then plans each rung from those cells, and a duel at the verdict regime decides whether the plan replaces the champion. What one model measures, the next model of its family inherits.
Standard uniform quantization applies one bitwidth family across the whole network. Paretrix reads the model's own response to each move (which classes are expensive to cut, which are nearly free, which pay back a buy), then spends the budget where the measurements show the greatest return.
🪜 The Mod-3 Ladder
Each rung lands on a multiple of 3 percent of the unquantized BF16 source, so the shipping name states its weight band (model-Paretrix-<Tier>-<XX>pc.gguf, ±1.5 pp). Every rung is planned from Q8_0 and from each stock base within ×0.85–1.10 of its weight; the last column is the ladder of OvisOCR3 (0.8B).
| Tier | Ratio | Planned from | Primary Purpose | KLD (OvisOCR3) |
|---|---|---|---|---|
| Fidelity | 48% | Q8_0 | High-fidelity archival tier. | 0.0025 |
| Precision | 42% | Q8_0 · Q6_K-imx | Coding, mathematics, high-entropy reasoning. | 0.0048 |
| Quality | 36% | Q8_0 · Q6_K-imx | General production tier. | 0.0152 |
| Compact | 33% | Q8_0 · IQ4_XS-imx | Daily driver for 2B–4B architectures. | 0.0310 |
| Mini | 30% | Q8_0 · IQ4_XS-imx | Balanced entry tier for ≥ 9B models on 8 GB VRAM. | 0.0544 |
| Nano | 27% | Q8_0 · IQ4_XS-imx | Compact edge deployment below stock IQ4_XS. |
0.0849 |
| Pico | 24% | Q8_0 | Edge budget. | 0.1702 |
| Femto | 21% | Q8_0 | Deepest service rung. | 0.2947 |
The recommended rung ⭐ is the lightest rung that clears a KLD ceiling at a top-p floor, both tightened as the model shrinks (small models have less redundancy to spend):
| Model size | KLD | Top-p |
|---|---|---|
| 4B and above | under 0.06 | 90% or more |
| 2B to under 4B | under 0.04 | 91.5% or more |
| Under 2B | under 0.02 | 93% or more |
The ladder names it at its close.
Every crown meets an opponent. Each plan duels the rung's champion at ctx 4096×8; a champion that holds receives one local pass (every class one notch up and down from where it stands); the run closes on the stock line, and a stock quant that dominates a champion, or ties it lighter, takes the rung through the same duel.
🧮 THE MATRIX — knowledge that transfers
Cells are stored in units of the model's sensitivity σ = KLD(IQ4_XS-imx) − KLD(Q8_0), so checkpoints of different sizes and fidelities share one scale. Three levels, each overlaying the next:
- Model cache (
model-paretrix.json): this model's matrices, plans, laws and champions — the one file a model shares, and the one a finetune can start from (a copied cache becomes a prior: its cells are re-measured on the new checkpoint). - Family line (
arch-paretrix/<arch>-paretrix.json): the mean cell of every checkpoint of themodel_type, with its range; checkpoints of a comparable size speak first. - Universal prior (
arch-paretrix/generic-paretrix.json): the mean of the family means, each family counted once.
The fold (Paretrix.py fold, at the close of every ladder) compiles every model cache under the suite root into those files; a checkpoint whose folder has left the root stays carried by the arch JSON's ledger. The next model of a known family plans with that knowledge and measures only the cells its plans sit on, and every cell a shipped plan uses is measured on the model itself. In σ units the FFN cells of NeoHorse (qwen35 GDN) and Spark (dense SWA) agree within ~10–20% while attention differs ×3–5: FFN prices transfer across families, attention stays a family law, and size moves the tied vocabulary (embd Q4_K: 1.04 σ at 0.8B, 0.12 σ at 4B).
⚔️ The engine
python Paretrix.py ladder ./MyModel # bases → matrices → plans → duels → polish → frontier → fold
python Paretrix.py ladder ./MyModel --rungs nano,mini --dry-run
python Paretrix.py duel --model model-BF16.gguf --tier quality \
--candidate model-Paretrix-Candidate-36pc.gguf --adopt
Per class, the measured cells form a lower convex hull in both directions (sells save MiB, buys spend them); options reach the least important tensors first (imatrix order), and an exact multiple-choice knapsack returns the least predicted KLD at the rung's weight under the model's composition law KLD − base ≈ b·Σ + e·(classes moved). Every plan is built by tensor reuse (only the moved tensors are quantized) and measured; a lighter plan inside the best plan's tie zone takes its place. The duel adopts on a KLD gain of at least max(0.001, 2 × the paired standard error of the chunk-wise deltas). Inside that gate, weight decides only when it differs by at least 0.5% of the file (PARETRIX_TWIN_MIN_PCT): the lighter twin wins if top-p holds, and a heavier candidate has no edge; at equal weight the better KLD point estimate takes the tie unless top-p vetoes. PPL never enters the verdict. Mechanism and measured facts: DOCUMENTATION.md §5–§7.
📊 Benchmarks
All sweeps run at the verdict regime ctx 4096×8 (32,768-token span) on the corpus wiki.test.raw with Flash-Attention. The KL reference is the BF16 logits of the same model. Compare KLD only within a single regime.
Models run from largest to smallest by BF16 size, and each table runs from largest to smallest MiB.
Pareto column: ★ = on this sweep's (MiB, KLD, top-p) frontier · < X = this row is dominated by X (X is lighter and better) · ≈ X = X is lighter, with KLD inside the tie zone max(MASD, 0.001). Bold KLD = a Paretrix rung that strictly dominates a stock quant · ⭐ = the recommended rung · † = PPL below the BF16 reference (over-confidence, not a quality gain).
1. ornith-ai/Ornith-1.5-9B — qwen35 GDN hybrid · untied 248k vocab
BF16 17,091 MiB · target: 8+ GB VRAM (split offload) · PPL reference 8.9214 · DFlash draft + MTP head, quantized as Paretrix-Balanced.
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 9086 | 8.9461 | +0.0247 | 0.0100 | 2.78% | 97.7% | ★ |
| Fidelity-48pc | 8212 | 8.9078† | −0.0136 | 0.0118 | 2.81% | 97.3% | ★ |
| Precision-42pc | 7325 | 8.8891† | −0.0323 | 0.0149 | 3.08% | 96.3% | ★ |
| Q6_K-imx (stock) | 7018 | 8.6805† | −0.2409 | 0.0240 | 3.85% | 95.7% | ★ |
| Quality-36pc | 6306 | 8.8716† | −0.0498 | 0.0291 | 4.26% | 94.2% | ★ |
| Q5_K_M-imx (stock) | 6168 | 8.2131† | −0.7083 | 0.1045 | 7.33% | 90.4% | < Compact-33pc |
| Compact-33pc | 5793 | 8.7547† | −0.1667 | 0.0460 | 5.66% | 92.6% | ★ |
| Mini-30pc ⭐ | 5281 | 8.8874† | −0.0340 | 0.0575 | 6.40% | 91.0% | ★ |
| IQ4_XS-imx (stock) | 4956 | 9.2634 | +0.3420 | 0.0767 | 6.92% | 90.9% | ★ |
| Nano-27pc | 4794 | 9.0223 | +0.1009 | 0.0773 | 6.84% | 90.1% | ★ |
| Pico-24pc | 4306 | 8.9920 | +0.0706 | 0.1263 | 8.84% | 86.6% | ★ |
| IQ3_M-imx (stock) | 4211 | 9.1423 | +0.2209 | 0.1806 | 10.94% | 84.6% | ★ |
| Femto-21pc | 3802 | 9.4044 | +0.4830 | 0.1810 | 11.24% | 83.5% | ★ |
Compact-33pc strictly dominates Q5_K_M-imx (−0.0585 KLD, −375 MiB); Mini-30pc ⭐ holds 91.0% top-p at 5281 MiB, ×3.2 smaller than BF16. Fidelity-48pc stays within 0.0018 KLD of Q8_0 for 874 MiB less.
2. TokenRhythm/NeoHorse-1-4B — qwen3.5 fine-tune · GDN hybrid · tied
BF16 8,034 MiB · target: 4+ GB VRAM · PPL reference 9.1272 · MTP head grafted on every rung (mtp/) · ladder from scratch in 2h35 (matrix 70 min, 51 of 103 cells).
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 4275 | 9.1380 | +0.0108 | 0.0040 | 1.69% | 98.4% | ★ |
| Fidelity-48pc | 3866 | 9.1000† | −0.0272 | 0.0060 | 2.17% | 97.7% | ★ |
| Precision-42pc | 3362 | 9.1840 | +0.0568 | 0.0081 | 2.43% | 97.0% | ★ |
| Q6_K-imx (stock) | 3304 | 9.2051 | +0.0779 | 0.0094 | 2.66% | 96.8% | ★ |
| Quality-36pc | 2963 | 9.1472 | +0.0200 | 0.0183 | 3.50% | 95.2% | ★ |
| Q5_K_M-imx (stock) | 2933 | 9.3837 | +0.2565 | 0.0302 | 4.42% | 94.4% | ★ |
| Compact-33pc | 2723 | 9.0879† | −0.0393 | 0.0271 | 4.35% | 93.8% | ★ |
| Mini-30pc | 2482 | 9.2485 | +0.1214 | 0.0408 | 5.47% | 91.7% | ★ |
| IQ4_XS-imx (stock) | 2398 | 9.5270 | +0.3999 | 0.0604 | 6.51% | 90.8% | ★ |
| Nano-27pc ⭐ | 2253 | 9.3778 | +0.2506 | 0.0592 | 6.35% | 90.2% | ★ |
| IQ3_M-imx (stock) | 2063 | 9.9012 | +0.7740 | 0.1476 | 10.60% | 85.0% | < Pico-24pc |
| Pico-24pc | 2024 | 9.2742 | +0.1470 | 0.1002 | 8.23% | 86.6% | ★ |
| Femto-21pc | 1792 | 9.7259 | +0.5987 | 0.1601 | 11.04% | 83.2% | ★ |
Pico-24pc strictly dominates IQ3_M-imx (−0.0475 KLD, −39 MiB); Nano-27pc ⭐ holds 90.2% top-p at 2253 MiB, ×3.6 smaller than BF16.
3. Nanbeige/Nanbeige4.2-3B — looped dense · untied 166k vocab
BF16 7,957 MiB · target: 6+ GB VRAM · PPL reference 29.9102 · DSpark draft, quantized as Paretrix-Balanced.
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 4229 | 31.6031 | +1.6930 | 0.0154 | 3.13% | 95.5% | ★ |
| Fidelity-48pc | 3822 | 31.5988 | +1.6887 | 0.0206 | 3.54% | 94.6% | ★ |
| Precision-42pc ⭐ | 3412 | 31.9853 | +2.0751 | 0.0345 | 4.30% | 92.5% | ★ |
| Q6_K-imx (stock) | 3266 | 31.9481 | +2.0380 | 0.0451 | 4.86% | 92.1% | ★ |
| Quality-36pc | 2932 | 31.9507 | +2.0405 | 0.0657 | 5.93% | 89.4% | ★ |
| Q5_K_M-imx (stock) | 2849 | 32.1389 | +2.2287 | 0.0858 | 6.52% | 88.0% | ★ |
| Compact-33pc | 2697 | 32.5806 | +2.6705 | 0.1105 | 7.65% | 86.9% | ★ |
| Mini-30pc | 2458 | 32.1243 | +2.2141 | 0.1443 | 8.38% | 84.4% | ★ |
| IQ4_XS-imx (stock) | 2268 | 32.9458 | +3.0356 | 0.2104 | 10.15% | 81.6% | < Nano-27pc |
| Nano-27pc | 2230 | 33.9472 | +4.0370 | 0.1982 | 9.70% | 81.8% | ★ |
| Pico-24pc | 2004 | 34.4681 | +4.5580 | 0.3044 | 12.10% | 76.5% | ★ |
| IQ3_M-imx (stock) | 1985 | 33.0494 | +3.1392 | 0.4926 | 15.17% | 71.2% | ★ |
| Femto-21pc | 1769 | 33.9181 | +4.0079 | 0.5080 | 15.42% | 68.5% | ★ |
Nano-27pc strictly dominates IQ4_XS-imx (−0.0122 KLD, −38 MiB); Precision-42pc ⭐ holds 92.5% top-p at 3412 MiB, ×2.3 smaller than BF16.
4. XHToken/Spark-X2.6-4B — dense SWA hybrid 3:1 · tied 131k vocab
BF16 7,849 MiB · target: 6+ GB VRAM · PPL reference 14.7611 · ladder from scratch in 1h30 (matrix 33 min, 39 of 74 cells) · DSpark draft, quantized as Paretrix-Balanced (×1.79) and rebuilt by 16 (draft-32K ×2.07 · draft-lite ×1.94).
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 4172 | 14.9008 | +0.1397 | 0.0037 | 1.43% | 97.7% | ★ |
| Fidelity-48pc | 3767 | 14.8424 | +0.0813 | 0.0068 | 1.90% | 97.0% | ★ |
| Precision-42pc | 3344 | 14.7933 | +0.0323 | 0.0125 | 2.85% | 95.6% | ★ |
| Q6_K-imx (stock) | 3223 | 14.8915 | +0.1304 | 0.0148 | 2.97% | 95.2% | ★ |
| Quality-36pc ⭐ | 2894 | 15.2484 | +0.4873 | 0.0278 | 4.30% | 93.0% | ★ |
| Q5_K_M-imx (stock) | 2840 | 15.8504 | +1.0893 | 0.0503 | 5.42% | 90.7% | ★ |
| Compact-33pc | 2660 | 15.2058 | +0.4447 | 0.0603 | 5.88% | 89.4% | ★ |
| Mini-30pc | 2424 | 15.2393 | +0.4783 | 0.0779 | 6.74% | 88.2% | ★ |
| IQ4_XS-imx (stock) | 2266 | 14.8916 | +0.1305 | 0.1241 | 8.23% | 84.9% | ≈ Nano-27pc |
| Nano-27pc | 2201 | 15.3033 | +0.5422 | 0.1230 | 8.32% | 85.0% | ★ |
| Pico-24pc | 1977 | 15.6347 | +0.8736 | 0.2219 | 11.62% | 79.7% | ★ |
| IQ3_M-imx (stock) | 1949 | 19.9445 | +5.1834 | 0.4527 | 16.89% | 72.2% | < Femto-21pc |
| Femto-21pc | 1751 | 17.8678 | +3.1067 | 0.3717 | 14.88% | 74.4% | ★ |
Femto-21pc strictly dominates IQ3_M-imx (−0.0810 KLD, −198 MiB); Nano-27pc ties IQ4_XS-imx 65 MiB lighter; Quality-36pc ⭐ holds 93.0% top-p at 2894 MiB, ×2.7 smaller than BF16.
5. webAI-Official/TwIL-LM3-Pro — granite dense · untied 128k vocab
BF16 6,984 MiB · target: 6+ GB VRAM · PPL reference 11.4981.
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 3712 | 11.5033 | +0.0052 | 0.0019 | 1.26% | 98.1% | ★ |
| Fidelity-48pc | 3354 | 11.4941† | −0.0039 | 0.0027 | 1.52% | 97.6% | ★ |
| Precision-42pc | 2996 | 11.5369 | +0.0388 | 0.0037 | 1.68% | 96.9% | ★ |
| Q6_K-imx (stock) | 2867 | 11.5066 | +0.0085 | 0.0053 | 1.96% | 96.5% | ★ |
| Quality-36pc | 2577 | 11.5803 | +0.0822 | 0.0084 | 2.49% | 95.6% | ★ |
| Q5_K_M-imx (stock) | 2493 | 11.6805 | +0.1824 | 0.0129 | 3.16% | 94.5% | ★ |
| Compact-33pc | 2367 | 11.6557 | +0.1577 | 0.0142 | 3.16% | 94.4% | ★ |
| Mini-30pc ⭐ | 2158 | 11.6760 | +0.1779 | 0.0219 | 3.88% | 92.8% | ★ |
| Nano-27pc | 1958 | 11.7715 | +0.2734 | 0.0309 | 4.53% | 91.4% | ★ |
| IQ4_XS-imx (stock) | 1937 | 11.8226 | +0.3245 | 0.0353 | 4.99% | 91.0% | ★ |
| Pico-24pc | 1758 | 12.0556 | +0.5576 | 0.0552 | 6.18% | 88.6% | ★ |
| IQ3_M-imx (stock) | 1653 | 12.4936 | +0.9956 | 0.1056 | 8.62% | 84.6% | < Femto-21pc |
| Femto-21pc | 1553 | 12.2507 | +0.7526 | 0.0989 | 8.13% | 84.4% | ★ |
Femto-21pc strictly dominates IQ3_M-imx (−0.0067 KLD, −100 MiB); Mini-30pc ⭐ holds 92.8% top-p at 2158 MiB, ×3.2 smaller than BF16. The whole ladder, Femto-21pc included, stays under 0.10 KLD.
6. openbmb/MiniCPM5-2B — dense llama-arch · untied 130k readout
BF16 4,806 MiB · target: 4+ GB VRAM · PPL reference 11.9600 · DSpark draft, quantized as Paretrix-Balanced · ladder from scratch in 53 min (matrix 22 min, 43 of 74 cells).
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 2556 | 11.9747 | +0.0147 | 0.0015 | 1.03% | 97.8% | ★ |
| Fidelity-48pc | 2309 | 12.0037 | +0.0437 | 0.0026 | 1.33% | 97.3% | ★ |
| Precision-42pc | 2060 | 12.0033 | +0.0432 | 0.0040 | 1.61% | 96.7% | ★ |
| Q6_K-imx (stock) | 1974 | 12.0143 | +0.0543 | 0.0059 | 1.88% | 95.9% | ★ |
| Quality-36pc | 1772 | 12.0571 | +0.0971 | 0.0108 | 2.63% | 94.6% | ★ |
| Q5_K_M-imx (stock) | 1724 | 12.1437 | +0.1837 | 0.0188 | 3.52% | 93.2% | ★ |
| Compact-33pc ⭐ | 1628 | 12.1659 | +0.2059 | 0.0189 | 3.57% | 92.8% | ★ |
| Mini-30pc | 1487 | 12.3445 | +0.3845 | 0.0362 | 4.88% | 90.1% | ★ |
| IQ4_XS-imx (stock) | 1358 | 12.4819 | +0.5219 | 0.0718 | 6.75% | 86.8% | < Nano-27pc |
| Nano-27pc | 1350 | 12.3905 | +0.4305 | 0.0551 | 5.94% | 88.3% | ★ |
| Pico-24pc | 1210 | 12.9191 | +0.9591 | 0.1029 | 8.15% | 84.1% | ★ |
| IQ3_M-imx (stock) | 1170 | 13.9163 | +1.9562 | 0.2029 | 11.88% | 78.5% | ★ |
| Femto-21pc | 1073 | 14.2053 | +2.2453 | 0.2073 | 11.88% | 78.0% | ★ |
Nano-27pc strictly dominates IQ4_XS-imx (−0.0167 KLD, −8 MiB); Compact-33pc ⭐ holds 92.8% top-p at 1628 MiB, ×3.0 smaller than BF16. Compact-33pc matches Q5_K_M-imx's KLD (0.0189 against 0.0188) while weighing 96 MiB less.
7. LiquidAI/LFM3-1.7B — short-conv + sliding/global attention hybrid · 128k vocab
BF16 3,306 MiB · target: 2+ GB VRAM · PPL reference 21.4464.
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 1761 | 21.4793 | +0.0329 | 0.0037 | 1.19% | 97.3% | ★ |
| Fidelity-48pc | 1594 | 21.0649† | −0.3815 | 0.0081 | 2.09% | 96.0% | ★ |
| Precision-42pc ⭐ | 1418 | 21.3036† | −0.1427 | 0.0100 | 2.19% | 95.3% | ★ |
| Q6_K-imx (stock) | 1362 | 21.1512† | −0.2952 | 0.0124 | 2.41% | 94.9% | ★ |
| Quality-36pc | 1218 | 21.0642† | −0.3821 | 0.0283 | 3.62% | 92.1% | ★ |
| Q5_K_M-imx (stock) | 1200 | 20.9749† | −0.4714 | 0.0415 | 4.37% | 90.6% | ★ |
| Compact-33pc | 1121 | 21.2786† | −0.1678 | 0.0553 | 4.82% | 89.0% | ★ |
| Mini-30pc | 1021 | 22.3830 | +0.9366 | 0.0972 | 6.58% | 85.0% | ★ |
| IQ4_XS-imx (stock) | 957 | 22.6938 | +1.2475 | 0.1512 | 8.37% | 81.5% | ≈ Nano-27pc |
| Nano-27pc | 927 | 22.8466 | +1.4002 | 0.1397 | 8.04% | 82.0% | ★ |
| Pico-24pc | 832 | 24.7068 | +3.2605 | 0.2104 | 10.08% | 78.3% | ★ |
| IQ3_M-imx (stock) | 829 | 21.1749† | −0.2714 | 0.2799 | 11.38% | 75.7% | ★ |
| Femto-21pc | 738 | 26.4226 | +4.9762 | 0.3574 | 13.04% | 71.3% | ★ |
Nano-27pc ties IQ4_XS-imx 31 MiB lighter; Precision-42pc ⭐ holds 95.3% top-p at 1418 MiB, ×2.3 smaller than BF16.
8. XHToken/Spark-X2.6-1.7B — dense SWA hybrid 3:1 · tied 131k vocab
BF16 3,262 MiB · target: 2+ GB VRAM · PPL reference 36.2154 · DSpark draft, quantized as Paretrix-Balanced (×1.40) and rebuilt by 16 (draft-32K ×1.58 · draft-lite ×1.55).
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 1736 | 36.5458 | +0.3304 | 0.0063 | 1.87% | 95.8% | ★ |
| Fidelity-48pc ⭐ | 1569 | 36.6322 | +0.4169 | 0.0170 | 2.85% | 93.4% | ★ |
| Precision-42pc | 1399 | 36.6095 | +0.3942 | 0.0247 | 3.53% | 91.5% | ★ |
| Q6_K-imx (stock) | 1341 | 36.0955† | −0.1199 | 0.0412 | 4.58% | 89.7% | ★ |
| Quality-36pc | 1202 | 35.2522† | −0.9632 | 0.0517 | 5.13% | 88.2% | ★ |
| Q5_K_M-imx (stock) | 1194 | 36.7500 | +0.5346 | 0.0840 | 6.11% | 86.0% | ★ |
| Compact-33pc | 1106 | 33.3808† | −2.8346 | 0.0984 | 6.83% | 83.8% | ★ |
| Mini-30pc | 1008 | 34.0220† | −2.1934 | 0.1352 | 7.74% | 81.4% | ★ |
| IQ4_XS-imx (stock) | 971 | 37.6937 | +1.4783 | 0.2728 | 11.24% | 74.9% | < Nano-27pc |
| Nano-27pc | 915 | 37.2969 | +1.0816 | 0.1961 | 9.38% | 77.7% | ★ |
| IQ3_M-imx (stock) | 848 | 53.1946 | +16.9793 | 0.5182 | 16.39% | 65.6% | < Pico-24pc |
| Pico-24pc | 821 | 45.7963 | +9.5810 | 0.3746 | 13.30% | 70.1% | ★ |
| Femto-21pc | 728 | 42.5738 | +6.3585 | 0.5611 | 16.23% | 64.4% | ★ |
Nano-27pc strictly dominates IQ4_XS-imx (−0.0767 KLD, −56 MiB); Pico-24pc strictly dominates IQ3_M-imx (−0.1436 KLD, −26 MiB); Fidelity-48pc ⭐ holds 93.4% top-p at 1569 MiB, ×2.1 smaller than BF16.
9. jinaai/ReaderLM-v2 — dense qwen2 · tied 152k vocab · HTML/Markdown extractor
BF16 2,950 MiB · target: 2+ GB VRAM · PPL reference 12.3827.
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 1570 | 12.4094 | +0.0268 | 0.0016 | 0.99% | 97.7% | ★ |
| Fidelity-48pc | 1421 | 12.4264 | +0.0437 | 0.0027 | 1.29% | 97.1% | ★ |
| Precision-42pc | 1265 | 12.3894 | +0.0068 | 0.0039 | 1.53% | 96.5% | ★ |
| Q6_K-imx (stock) | 1214 | 12.3816† | −0.0011 | 0.0053 | 1.83% | 96.1% | ★ |
| Quality-36pc ⭐ | 1088 | 12.4294 | +0.0467 | 0.0119 | 2.74% | 93.8% | ★ |
| Q5_K_M-imx (stock) | 1073 | 12.4151 | +0.0324 | 0.0163 | 3.22% | 93.2% | ★ |
| Compact-33pc | 1000 | 12.4528 | +0.0702 | 0.0218 | 3.71% | 92.4% | ★ |
| Mini-30pc | 911 | 12.4957 | +0.1131 | 0.0366 | 4.74% | 90.1% | ★ |
| IQ4_XS-imx (stock) | 854 | 12.5477 | +0.1651 | 0.0580 | 5.94% | 87.6% | < Nano-27pc |
| Nano-27pc | 827 | 12.5672 | +0.1846 | 0.0530 | 5.76% | 88.0% | ★ |
| Pico-24pc | 743 | 12.9184 | +0.5358 | 0.0988 | 7.67% | 84.0% | ★ |
| IQ3_M-imx (stock) | 741 | 13.3858 | +1.0031 | 0.1706 | 10.39% | 79.6% | ★ |
| Femto-21pc | 659 | 13.5861 | +1.2034 | 0.1816 | 10.63% | 78.8% | ★ |
Nano-27pc strictly dominates IQ4_XS-imx (−0.0050 KLD, −27 MiB); Quality-36pc ⭐ holds 93.8% top-p at 1088 MiB, ×2.7 smaller than BF16.
10. ATH-MaaS/OvisOCR3 — qwen35 GDN hybrid · tied 248k vocab · OCR
BF16 1,446 MiB · target: 2+ GB VRAM · PPL reference 29.1182.
| Model | MiB | PPL | ΔPPL | KLD | RMS Δp | top-p | Pareto |
|---|---|---|---|---|---|---|---|
| Q8_0 (stock) | 774 | 29.2935 | +0.1753 | 0.0015 | 0.93% | 97.9% | ★ |
| Fidelity-48pc | 704 | 29.4017 | +0.2836 | 0.0025 | 1.19% | 97.2% | ★ |
| Precision-42pc | 620 | 29.2790 | +0.1608 | 0.0048 | 1.63% | 95.9% | ★ |
| Q6_K-imx (stock) | 601 | 29.5624 | +0.4442 | 0.0060 | 1.76% | 95.5% | ★ |
| Q5_K_M-imx (stock) | 551 | 30.0810 | +0.9628 | 0.0154 | 2.90% | 93.3% | ★ |
| Quality-36pc ⭐ | 533 | 29.9233 | +0.8051 | 0.0152 | 2.96% | 93.1% | ★ |
| Compact-33pc | 490 | 30.9356 | +1.8174 | 0.0310 | 4.32% | 90.5% | ★ |
| IQ4_XS-imx (stock) | 481 | 33.3879 | +4.2697 | 0.0592 | 5.82% | 87.6% | < Mini-30pc |
| Mini-30pc | 446 | 31.5577 | +2.4396 | 0.0544 | 5.40% | 86.9% | ★ |
| IQ3_M-imx (stock) | 433 | 34.9254 | +5.8072 | 0.1561 | 9.90% | 81.4% | < Nano-27pc |
| Nano-27pc | 405 | 32.3247 | +3.2065 | 0.0849 | 6.76% | 84.8% | ★ |
| Pico-24pc | 364 | 38.9024 | +9.7842 | 0.1702 | 9.86% | 77.9% | ★ |
| Femto-21pc | 323 | 44.3950 | +15.2768 | 0.2947 | 12.96% | 71.6% | ★ |
Mini-30pc strictly dominates IQ4_XS-imx (−0.0048 KLD, −34 MiB); Nano-27pc strictly dominates IQ3_M-imx (−0.0712 KLD, −28 MiB); Quality-36pc ⭐ holds 93.1% top-p at 533 MiB, ×2.7 smaller than BF16.
📉 Sub-Nano compendium — the Nano → Pico → Femto steps
Families run from largest to smallest. Reading grid at the verdict regime: 🟢 solid (KLD under 0.10) · 🟡 usable (0.10–0.20) · 🔴 degraded (over 0.20).
| Family | Architecture | Nano KLD | Pico KLD | Femto KLD |
|---|---|---|---|---|
| Ornith-1.5-9B | GDN hybrid, untied | 0.0773 🟢 | 0.1263 🟡 | 0.1810 🟡 |
| NeoHorse-1-4B | GDN fine-tune, tied | 0.0592 🟢 | 0.1002 🟡 | 0.1601 🟡 |
| Nanbeige4.2-3B | Looped dense, untied | 0.1982 🟡 | 0.3044 🔴 | 0.5080 🔴 |
| Spark-X2.6-4B | SWA hybrid, tied | 0.1230 🟡 | 0.2219 🔴 | 0.3717 🔴 |
| TwIL-LM3-Pro | Dense, untied | 0.0309 🟢 | 0.0552 🟢 | 0.0989 🟢 |
| MiniCPM5-2B | Dense, untied | 0.0551 🟢 | 0.1029 🟡 | 0.2073 🔴 |
| LFM3-1.7B | Conv/attention hybrid | 0.1397 🟡 | 0.2104 🔴 | 0.3574 🔴 |
| Spark-X2.6-1.7B | SWA hybrid, tied | 0.1961 🟡 | 0.3746 🔴 | 0.5611 🔴 |
| ReaderLM-v2 | Dense, tied | 0.0530 🟢 | 0.0988 🟢 | 0.1816 🟡 |
| OvisOCR3 | GDN hybrid, tied | 0.0849 🟢 | 0.1702 🟡 | 0.2947 🔴 |
Summary — strict Pareto dominations
11 products are at most as heavy as their stock neighbour and better beyond the tie zone, at the verdict regime. Families run from largest to smallest; tiers run from largest to smallest within each family.
| Family | Product | KLD | Stock twin | Stock KLD | ΔKLD | ΔMiB |
|---|---|---|---|---|---|---|
| Ornith-1.5-9B | Compact-33pc | 0.0460 | Q5_K_M-imx | 0.1045 | −0.0585 | −375 |
| NeoHorse-1-4B | Pico-24pc | 0.1002 | IQ3_M-imx | 0.1476 | −0.0475 | −39 |
| Nanbeige4.2-3B | Nano-27pc | 0.1982 | IQ4_XS-imx | 0.2104 | −0.0122 | −38 |
| Spark-X2.6-4B | Femto-21pc | 0.3717 | IQ3_M-imx | 0.4527 | −0.0810 | −198 |
| TwIL-LM3-Pro | Femto-21pc | 0.0989 | IQ3_M-imx | 0.1056 | −0.0067 | −100 |
| MiniCPM5-2B | Nano-27pc | 0.0551 | IQ4_XS-imx | 0.0718 | −0.0167 | −8 |
| Spark-X2.6-1.7B | Nano-27pc | 0.1961 | IQ4_XS-imx | 0.2728 | −0.0767 | −56 |
| Spark-X2.6-1.7B | Pico-24pc | 0.3746 | IQ3_M-imx | 0.5182 | −0.1436 | −26 |
| ReaderLM-v2 | Nano-27pc | 0.0530 | IQ4_XS-imx | 0.0580 | −0.0050 | −27 |
| OvisOCR3 | Mini-30pc | 0.0544 | IQ4_XS-imx | 0.0592 | −0.0048 | −34 |
| OvisOCR3 | Nano-27pc | 0.0849 | IQ3_M-imx | 0.1561 | −0.0712 | −28 |
Plus 2 products that tie their stock neighbour inside the tie zone while weighing less: Spark-X2.6-4B Nano-27pc (65 MiB lighter than IQ4_XS-imx), LFM3-1.7B Nano-27pc (31 MiB lighter than IQ4_XS-imx). No rung of any ladder is a renamed stock quant: every one is its own build.
⚡ Speculative decoding: MTP · DSpark · DFlash
Paretrix quantizes speculative modules imatrix-free (Paretrix-modules.py). A draft's input is the parent's hidden state, the MTP head rides the trunk's activations, and the vision tower consumes raw pixels, so none of them exposes a corpus entry point. Speculative verification bounds draft damage to the acceptance rate and leaves output fidelity intact. The Paretrix-Balanced quality matches the performance of the BF16 draft, and edges slightly ahead of it.
| Module | Role | Palette | Naming |
|---|---|---|---|
| MTP | Multi-token prediction fusion head (extracted by 01) | Role floors + K-quants | mtp-Paretrix-<Profile>.gguf |
| DSpark | Block-diffusion speculative draft (Markov + confidence heads) | Role floors + K-quants | dspark-Paretrix-<Profile>.gguf · rebuilt by 16: draft-<tag>/dspark-<tag>-Paretrix-Balanced.gguf |
| DFlash | Third-party block-diffusion draft | Role floors + K-quants | dflash-Paretrix-<Profile>.gguf |
| MMProj | CLIP vision projector | Profile grid, critical tensors pinned F32 | mmproj-Paretrix-<Profile>.gguf |
# Acceptance battery (llama-server A/B, 16K context prefill, q8_0/q5_1 KV salvos, chat template auto-detected)
# Acceptance belongs to the (draft, target) pair: compare at matched --ngl and KV type
python 13_draft-acceptance-sweep.py <working-folder>
python 13_draft-acceptance-sweep.py model.gguf draft.gguf --n-max 3,7,15
# Fuse model + modules into one deployment GGUF
python 14_gguf-module-fusion.py fused.gguf model-BF16.gguf mmproj-BF16.gguf
# Graft the MTP head onto every Paretrix rung → Paretrix/mtp/model-mtp-Paretrix-<Tier>-<XX>pc.gguf
python 14_gguf-module-fusion.py MyModel/Paretrix mtp-Paretrix-Balanced.gguf
🧬 DSpark drafts rebuilt — 16_dspark-generator.py
An official DSpark draft re-reads the target's whole vocab head on every draft pass. 16 gives it a d2t readout of its own, the 32,768 tokens the target actually generates (99.45% of them), with no training and the same logits on those tokens (draft-32K); or prunes it, keeps that readout and trains it for a night (draft-lite, --lite); or trains a Lite draft from the target's own layers when no official draft exists (home mode). --32k (the default) to --8k set the readout: on an agent's long contexts, 32K holds the balance. Every draft is quantized by Paretrix-modules.py (Balanced), measured by 13 at each n_max, and shipped in its own folder with the llama-server flags of the fastest. The official draft can be given as its HF folder: 16 converts it first.
| Target | Draft | Module | Speedup |
|---|---|---|---|
| Spark-X2.6-4B | official draft | 415 MiB | ×1.79 |
| Spark-X2.6-4B | draft-32K · official layers + own readout |
453 MiB | ×2.07 |
| Spark-X2.6-4B | draft-lite · official pruned and trained |
209 MiB | ×1.94 |
| Spark-X2.6-1.7B | official draft | 232 MiB | ×1.40 |
| Spark-X2.6-1.7B | draft-32K · official layers + own readout |
259 MiB | ×1.58 |
| Spark-X2.6-1.7B | draft-lite · official pruned and trained |
143 MiB | ×1.55 |
Every draft quantized Paretrix-Balanced; 13 at ctx 4096, target and draft KV f16, 24 thinking probes, RTX 4060 8 GB, n_max 5 (the best for every draft); each speedup against the baseline of its own sweep.
python 16_dspark-generator.py MyModel/Paretrix/model-Paretrix-Precision-42pc.gguf MyModel-DSpark # convert (≈ 10 min)
python 16_dspark-generator.py MyModel/Paretrix/model-Paretrix-Precision-42pc.gguf official.gguf --lite # a night
python 16_dspark-generator.py MyModel/Paretrix/model-Paretrix-Precision-42pc.gguf # home mode
🤖 Sub-agent profile — 15_subagent-profile-bench.py
15 judges a GGUF as the sub-agent a larger hosted model delegates to: finding a constant in 100K tokens, abstaining on names that exist nowhere, mapping callers, copying a block verbatim, predicting programs and shell one-liners, computing, converting formats, reading images. Every answer is scored by code (AST sets, exact keys, executed programs; no judge model), and a registry row per configuration opens roles (scout, trusted, cartographer, extractor, documentalist…). The arms compare the target alone, with its MTP head, or with a draft found beside it or in its draft-*/ folders.
On Spark-X2.6-1.7B Precision-42pc (windows of 131K to 524K tokens, KV q5_1), the draft-lite DSpark doubles the decoding speed of the whole battery: 47.6 → 98.9 t/s (acceptance 57.3%), with no invention on the 20 absent-element questions in either arm. On Spark-X2.6-4B Quality-36pc, draft-32K decodes 3.1–3.3× faster than the target alone at every tier (25.4 → 81.7 t/s at 131K, acceptance 54.6%) and draft-lite 2.4–2.6×; on 8 GB both hold the 196K window that reads the whole repository corpus, draft-32K at -ub 256. A draft changes the speed, never the answers: the target verifies every token. Each size takes its own place: Spark-X2.6-1.7B Fidelity-48pc opens scout (the deep needle held at 247K tokens in a 524K window; every function of the 181K-token repository corpus located and copied, at 51 t/s), a scout that reports on a whole repository; Spark-X2.6-4B Quality-36pc reaches 9.61/10 over 19 categories with trusted, cartographer, mathematician and transcriber, an analyst and verifier.
python 15_subagent-profile-bench.py verify # offline integrity of the answer keys, no model
python 15_subagent-profile-bench.py MyModel/Paretrix/model-Paretrix-Precision-42pc.gguf
python 15_subagent-profile-bench.py --aggregate # registry: configurations, matrix, roles
⚡ Quick start
# One command: convert → imatrix → stock line → the Paretrix ladder → fold
python 00_Paretrix-pipeline.py ./MyModel
# Or step by step
python 01_SAFETENSORS-to-BF16-GGUF.py ./MyModel # → Paretrix/model-BF16.gguf
python 02_BF16-GGUF-to-Q8-imatrix.py # → model-Q8_0.gguf + imatrix.gguf
python 03_BF16-GGUF-to-Paretrix.py ./MyModel # → stock line + every rung planned and dueled
# Selected rungs, plans only
python 03_BF16-GGUF-to-Paretrix.py ./MyModel --tier nano,mini --dry-run
# Also the comparison classics Q5_K_M-imx and IQ3_M-imx for 11 (the bases Q6_K-imx and IQ4_XS-imx are always built)
python 00_Paretrix-pipeline.py ./MyModel --all-classics
# Sweep every product at the verdict regime
python 11_perplexity-test.py
# A DSpark draft for the model (16), its acceptance (13), its sub-agent profile (15)
python 16_dspark-generator.py MyModel/Paretrix/model-Paretrix-Precision-42pc.gguf MyModel-DSpark
python 15_subagent-profile-bench.py MyModel/Paretrix/model-Paretrix-Precision-42pc.gguf
# What the suite knows about a model · its champions against their plans
python Paretrix.py show ./MyModel
python Paretrix.py verify ./MyModel
# Fold every model cache into arch-paretrix (the ladder does it at its close)
python Paretrix.py fold
# What a finished run leaves behind (list; --run removes; --publish adds the harness caches before a share)
python Paretrix.py tidy ./MyModel --publish
Working-root convention. A model folder holds its HF repo (safetensors + sidecars) plus a Paretrix/ subfolder carrying every product: BF16/Q8/imatrix GGUFs, the stock line, champions, modules, and model-paretrix.json. Every script finds the llama.cpp binaries in llama-cpp/build/bin/Release first, then at the root of llama-cpp/ (LLAMA_CPP_DIR and LLAMA_<NAME>_PATH override both).
<model folder>/
├─ model-00001-of-00002.safetensors · config.json · tokenizer.json …
└─ Paretrix/
├─ model-BF16.gguf · model-Q8_0.gguf · imatrix.gguf · model-<Q6_K|IQ4_XS>-imx.gguf (+ Q5_K_M, IQ3_M with --all-classics)
├─ model-Paretrix-<Tier>-<XX>pc.gguf (+ .provenance.json) · mtp/mmproj/dspark products
├─ draft-official/ · draft-32K/ · draft-lite/ (16: official BF16 drafts and the rebuilt DSpark drafts)
├─ model-paretrix.json (matrices, plans, laws, champions: the file to share)
├─ mtp/model-mtp-Paretrix-<Tier>-<XX>pc.gguf (14 graft: the rung + its MTP head)
└─ _transient/ · _logs/ · kld-bf16-ctx<N>.dat · eval-cache.json (harness caches; tidy --publish)
Full operational detail: DOCUMENTATION.md.
📜 Citation & Credits
- Paretrix Quantization Suite: measured per-class matrices, an exact multiple-choice knapsack under a fitted composition law, duel-gated adoption, and a cross-architecture knowledge base (THE MATRIX).
- llama.cpp by Georgi Gerganov & ggml contributors: GGUF/GGML runtime,
llama-imatrix,llama-quantize,llama-perplexity, andllama-server. - GSQ / RCO (IST-DASLab): GSQ: Gumbel-STE quantization (arXiv:2604.18556) · RCO: relaxed knapsack allocation (arXiv:2605.00649). The exact-budget allocation builds on these lines, adapted to measured cells.
- wepiqx ASHQ1: priority-queue knapsack formulation, tied-group activation hashing, and MSE scheduling.
- Empero AI: GDN state preservation (
ssm_alpha/ssm_betaat Q8_0). - XHToken: the Spark-X2.6 models, their official DSpark drafts (4B · 1.7B), the layers that
16_dspark-generator.pyconverts and prunes, and the llama.cpp fork with Spark2.5 and DSpark support (thedflashgraph anddraft-dsparkspeculation, merged here onto llama.cpp b12020), whose runtime contract 16 reproduces literally. - rasyosef: train-dspark-draft-models and K2-Horizon-3.7B-DSpark, a DSpark draft trained for a model of its own: the lead that the trained drafts of 16 (lite and home modes) follow.
- Nanbeige: Nanbeige4.2-3B-DSpark, the q/k-norm SiLU layout of the dflash drafts and the HF-folder form that 16 converts.
- mlabonne: open-perfectblend, the prompt mix behind the on-policy trajectories of 16.
- EAGLE-3 (SafeAILab): EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test (arXiv:2503.01840). Its reduced draft vocabulary, mapped to the target's ids by a
d2ttable, is the readout that 16 gives the DSpark drafts (draft-32K,draft-lite); the teacher and the trajectories of 16 come from the suite's EAGLE-3 pipeline, whose draft serves as the control of--diag.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support