Fair point. This targets boxes with no GPU, and 9 to 15 tok/s is well under your 45 to 60 bar. Your setup already beats it. The README's hybrid CPU plus GPU result is only for 30B models, not the 284B one, so for agent work a smaller MoE looks like the closer fit.
Sajjad Pakdamansavoji
savoji
AI & ML interests
VLM, VLA, LLM, Robotics, Computer Vision
Recent Activity
repliedto their post about 4 hours ago
A 284B-parameter DeepSeek-V4-Flash now runs on a 64 GB desktop with 0 GPUs: llama-modchip shrinks the checkpoint to 56.56 GiB (1.71 bpw) and decodes at 9.1 to 14.7 tok/s.
The standard UD-IQ1_S build is 76.87 GiB, above the ~62.5 GiB usable on a 64 GB Linux box. The repo mixes precisions (routed experts mostly IQ1_S, MLA projections in a custom Fractal-64 type) and patches the CPU kernels. The model plus a 256K KV cache then sits at 57.61 to 58.9 GiB RSS with zero swap. The repo was created on 2026-10-29 and already has 744 stars and 470 points on Hacker News.
š The numbers
⢠Size: 56.56 GiB, 20.31 GiB smaller than UD-IQ1_S
⢠Prefill: 25.0 to 44.36 tok/s (+77%) on a 16-core Ryzen 9 9955HX
⢠Decode: 5.91 tok/s unpatched, 9.06 to 9.95 tok/s at Top-6, 11.5 to 14.7 tok/s with the n-gram drafter
ā ļø Caveats
⢠"Lossless" means bit-exact against the same re-quantized checkpoint. Versus UD-IQ1_S the author reports 0.9964 top-logit cosine similarity
⢠The 98.5/100 quality score comes from the author's own gate, and the headline speeds come from one machine
⢠The custom tensor type 40 is rejected by stock llama.cpp, Ollama and LM Studio
š ļø Try it
⢠Run scripts/check_cpu_readiness.py from the repo: it needs no model download
⢠Run ./modchip bench for speed, then ./modchip eval to check the quality claims yourself
Repo:
https://github.com/synchopate/llama-modchip
HN thread:
https://news.ycombinator.com/item?id=50411961
Would you accept a custom runtime for 9 to 15 tok/s from a 284B model on a 64 GB box?
š Read it elsewhere
⢠The deep dive on Medium: https://medium.com/p/e47ab3581f58
⢠The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/llama-modchip-runs-a-284b-deepseek
⢠The thread on X: https://x.com/saavoji/status/2119858773280141526
⢠Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#llama-modchip repliedto their post about 4 hours ago
A 284B-parameter DeepSeek-V4-Flash now runs on a 64 GB desktop with 0 GPUs: llama-modchip shrinks the checkpoint to 56.56 GiB (1.71 bpw) and decodes at 9.1 to 14.7 tok/s.
The standard UD-IQ1_S build is 76.87 GiB, above the ~62.5 GiB usable on a 64 GB Linux box. The repo mixes precisions (routed experts mostly IQ1_S, MLA projections in a custom Fractal-64 type) and patches the CPU kernels. The model plus a 256K KV cache then sits at 57.61 to 58.9 GiB RSS with zero swap. The repo was created on 2026-10-29 and already has 744 stars and 470 points on Hacker News.
š The numbers
⢠Size: 56.56 GiB, 20.31 GiB smaller than UD-IQ1_S
⢠Prefill: 25.0 to 44.36 tok/s (+77%) on a 16-core Ryzen 9 9955HX
⢠Decode: 5.91 tok/s unpatched, 9.06 to 9.95 tok/s at Top-6, 11.5 to 14.7 tok/s with the n-gram drafter
ā ļø Caveats
⢠"Lossless" means bit-exact against the same re-quantized checkpoint. Versus UD-IQ1_S the author reports 0.9964 top-logit cosine similarity
⢠The 98.5/100 quality score comes from the author's own gate, and the headline speeds come from one machine
⢠The custom tensor type 40 is rejected by stock llama.cpp, Ollama and LM Studio
š ļø Try it
⢠Run scripts/check_cpu_readiness.py from the repo: it needs no model download
⢠Run ./modchip bench for speed, then ./modchip eval to check the quality claims yourself
Repo:
https://github.com/synchopate/llama-modchip
HN thread:
https://news.ycombinator.com/item?id=50411961
Would you accept a custom runtime for 9 to 15 tok/s from a 284B model on a 64 GB box?
š Read it elsewhere
⢠The deep dive on Medium: https://medium.com/p/e47ab3581f58
⢠The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/llama-modchip-runs-a-284b-deepseek
⢠The thread on X: https://x.com/saavoji/status/2119858773280141526
⢠Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#llama-modchip posted an update about 5 hours ago
Comfy-Org's repackaged MiniMax-H3-Image has 351,803 downloads last month, more than the 142,989 of the official repo, which was created five days earlier.
@MiniMaxAI's 33.1B dense DiT generates about 16 MP directly (4096x4096 up to 6272x2688), with no upscaler, in 6 steps and no classifier-free guidance. One checkpoint covers text-to-image and editing with up to 16 reference images. The card lists recipes for SGLang, vLLM-Omni, diffusers and ComfyUI, and @Comfy-Org added quantized files and workflow templates within days.
š What's new
⢠The H3 design without audio layers, 50 layers, with the H3 Qwen3-VL text encoder unchanged
⢠Only 6 steps and guidance 1 are accepted; other values return HTTP 400 and negative_prompt is rejected
š ļø Try it
⢠Official BF16 DiT is 68.65 GB, plus a separate 66.7 GB text encoder
⢠Comfy-Org offers a pruned int8 DiT, int8 or NVFP4 text encoders, an fp16 VAE, and T2I and Edit templates
⢠It expects a detailed English prompt: use the MiniMax API or the local MiniMax-H3-Image-PE (17.9 GB, NVFP4)
ā ļø Caveats
⢠The card gives no benchmark scores; "strongest instruction following among open-source models" is a vendor claim
⢠Community license: written authorization is needed above 1 million USD annualized revenue or for hosted services
https://huggingface.co/MiniMaxAI/MiniMax-H3-Image
https://huggingface.co/Comfy-Org/MiniMax-H3-Image
https://huggingface.co/spaces/MiniMaxAI/minimax-h3-image
Which route would you take for a local run: ComfyUI with the quantized files, or SGLang, vLLM-Omni or diffusers with the official weights?
š Read it elsewhere
⢠The deep dive on Medium: https://medium.com/p/f338de809dcd
⢠The long read on Substack: https://sajjadpakdamansavoji.substack.com/p/minimax-h3-image-is-a-16-mp-base
⢠The thread on X: https://x.com/saavoji/status/2120585989798207554
⢠Every version on sajjadpsavoji.github.io: https://sajjadpsavoji.github.io/blog#minimax-h3-imageOrganizations
None yet