Instructions to use netprojects/Inkling-2-Mini-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use netprojects/Inkling-2-Mini-MLX-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("netprojects/Inkling-2-Mini-MLX-4bit") config = load_config("netprojects/Inkling-2-Mini-MLX-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use netprojects/Inkling-2-Mini-MLX-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "netprojects/Inkling-2-Mini-MLX-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use netprojects/Inkling-2-Mini-MLX-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default netprojects/Inkling-2-Mini-MLX-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use netprojects/Inkling-2-Mini-MLX-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Inkling-2-Mini-MLX-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "netprojects/Inkling-2-Mini-MLX-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Inkling 2 Mini for Apple Silicon (MLX, 4-bit, + DSpark)
✅ Works with stock mlx-vlm (main now, next release) · OpenAI tool calls via inkling2-mlx
thinkingmachines/Inkling-2-Mini (revision 6e7c824) for MLX on
Apple Silicon, with its DSpark speculative-decoding draft bundled (mtp.safetensors) and its vision encoder included.
Everything Thinking Machines released in compressed form, plus attention, embeddings and the output layer, is re-quantized to plain affine 4-bit (group 64); the vision encoder stays as released. This is the plain-4-bit option; the 4/8-bit build keeps the released expert values exactly. Size: 16.5 GB plus the 0.64 GB draft.
Inkling 2 Mini is Thinking Machines' looped mixture-of-experts model: 26.4B parameters, about 3.4B active per token, 24 layers of which the middle twelve run three times (48 layer passes per token), text and image input, up to 1M tokens of context.
Builds
| Build | Size | Same next token as Thinking Machines' code | KL vs their code | GSM8K (50 questions) | Peak memory, 8K-token prompt |
|---|---|---|---|---|---|
| netprojects/Inkling-2-Mini-MLX-4-8bit (recommended) | 19.5 GB + 0.64 GB draft | 98.4% | 0.0018 | 47/50 | 27.8 GB |
| netprojects/Inkling-2-Mini-MLX-3bit | 14.1 GB + 0.64 GB draft | 95.5% | 0.0424 | 48/50 | 21.2 GB |
| netprojects/Inkling-2-Mini-MLX-2.5bit | 10.8 GB + 0.64 GB draft | 87.5% | 0.1899 | 45/50 | 18.2 GB |
| netprojects/Inkling-2-Mini-MLX-4bit | 16.5 GB + 0.64 GB draft | 95.9% | 0.0291 | 47/50 | 23.5 GB |
"Thinking Machines' code" is their Transformers implementation of Inkling 2 Mini, run on the released weights; the columns are explained under "What was checked".
Install
Inkling 2 Mini is supported in MLX-VLM itself since 21 November 2026 (not yet in a PyPI release). Use MLX-VLM from main (Python 3.10 or newer):
pip install "git+https://github.com/Neywa-Labs/mlx-vlm" torch torchvision
or, to also get OpenAI tool_calls from mlx_vlm.server, internet-projects/inkling2-mlx (the same MLX-VLM plus
Inkling's tool-call parser):
pip install "git+https://github.com/internet-projects/inkling2-mlx@v0.2.3" torch torchvision
PyTorch and torchvision are only used on the CPU by the image processor. On 21 November 2026 this repository's weight files were updated to the tensor names MLX-VLM uses (a rename of the loop-gate tensors; the values are unchanged): if you downloaded this build before, download it again.
Use
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("netprojects/Inkling-2-Mini-MLX-4bit")
prompt = apply_chat_template(processor, model.config, "Explain in two sentences why the sky is blue.",
reasoning_effort="low")
print(generate(model, processor, prompt, max_tokens=2048, temperature=0.0).text)
reasoning_effort is part of Inkling's chat template: none, minimal, low, medium, high (the default), xhigh,
max, or a number from 0.0 to 0.99. Thinking Machines notes that the model can think at length even at none and can get
stuck in long reasoning loops, so set max_tokens.
With an image:
prompt = apply_chat_template(processor, model.config, "What error does this terminal show?", num_images=1, reasoning_effort="low")
print(generate(model, processor, prompt, image=["screenshot.png"], max_tokens=2048).text)
With the DSpark draft (greedy output stays the model's own, up to near-ties):
from mlx_vlm.speculative.drafters import load_drafter
draft, kind = load_drafter("netprojects/Inkling-2-Mini-MLX-4bit")
print(generate(model, processor, prompt, max_tokens=2048, draft_model=draft, draft_kind=kind,
draft_block_size=4).text)
Server: mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-4bit --draft-model netprojects/Inkling-2-Mini-MLX-4bit --draft-kind dflash --draft-block-size 4.
OpenAI-compatible server, with tool calls and the thinking returned separately (reasoning_content):
mlx_vlm.server --model netprojects/Inkling-2-Mini-MLX-4bit --port 8000
Speed
MacBook Pro M5 Max (128 GB), idle, 21 November 2026; median of 3 runs. Decode: 5 prompts (math, code, chat) × 256 tokens, greedy, reasoning none. DSpark: the bundled draft at its best block size.
| Build | Prompt processing, 1K / 8K / 32K tokens (tok/s) | Generation (tok/s) | Generation with DSpark (tok/s) |
|---|---|---|---|
| 4-8bit | 1752 / 1658 / 1208 | 36.6 | 47.1 (×1.29, block 3) |
| 4bit (this build) | 1535 / 1412 / 1104 | 43.6 | 70.9 (×1.63, block 4) |
| 3bit | 1386 / 1351 / 1075 | 44.4 | 75.2 (×1.69, block 4) |
| 2.5bit | 1482 / 1384 / 1100 | 45.7 | 83.9 (×1.84, block 4) |
The draft's default block size (8) is slower on this model; use 3 or 4 as below.
With inkling2-mlx v0.2.3 (faster one-token decode, on by default; same answers within bf16 rounding), plain generation on the 4-bit build went from 41.0 to 44.6 tok/s with a 1K prompt and from 36.4 to 41.7 tok/s with a 16K prompt, and DSpark (block 4) at 16K from 85.4 to 90.9 tok/s (one prompt, 128 tokens, greedy, 22 November 2026; a separate run from the table above). INKLING2_FAST_DECODE=0 switches back.
Thinking Machines' results
From the Inkling 2 Mini card, for the original model (their harnesses and settings; a selection of their table):
| Benchmark (Thinking Machines' figures) | Inkling 2 Mini | Qwen3.8-27B | Gemma 4 31B |
|---|---|---|---|
| Artificial Analysis Intelligence Index v5.0 | 20.1 | 20.1 | 4.4 |
| SWE-bench Verified (bash only) | 79.2% | 77.8% | 46.1% |
| SWE-bench Pro v2 Hard (mini-swe-agent) | 77.9% | 73.8% | 14.2% |
| Terminal-Bench 2.1 (pi) | 71.6% | 73.0% | 43.4% |
| BrowseComp (with compaction) | 79.0% | 61.6% | 37.0% |
| MCP Atlas | 82.7% | 77.3% | 54.2% |
| MMMU Pro | 74.9% | 76.3% | 73.4% |
| OmniDocBench v1.6 | 92.8% | 91.5% | 82.5% |
| LongBench v2 | 66.9% | 52.3% | 52.9% |
| ARC-AGI v1 / v2 | 84.5 / 49.3% | 87.5 / 42.4% | – |
What was checked
On a MacBook Pro M5 Max (128 GB), 20–21 November 2026, with MLX 0.32.4. The comparison with Thinking Machines' code was re-run on 21 November with MLX-VLM main (fd6404f, as in inkling2-mlx v0.2.0); the other checks and the speeds below were measured with our earlier Inkling code (inkling2-mlx v0.1.0) on the same weights.
- Against Thinking Machines' code. Their Transformers implementation (pull request #51050) ran on the CPU with the released weights, on 12 short chat texts (488 answer positions). This build gave the same next token at 95.9% of positions (mean KL 0.0291), and its greedy answers to 3 prompts matched theirs for the first 4, 24, 24 of 24 tokens. Against our 4/8-bit build: 96.3% same next token, mean KL 0.0267.
- GSM8K, 50 fixed test questions (reasoning
none, greedy): 47/50. - Tool calling (reasoning
none): an agent prompt withbashandreadtools produced a well-formed call in Inkling's format, whichmlx_vlm.serverreturns as OpenAItool_calls. - Coding: 3/3 short Python tasks passed their asserts. Thinking (reasoning
low): 2/2 questions answered correctly after one closed thinking block. Vision: named the red square and blue circle of a test image. - Memory: peak 22.6 GB with a 4K-token prompt and 23.5 GB with an 8K-token prompt (mlx-vlm defaults). With a smaller prefill step the peak drops: 17.9 / 18.2 GB at
prefill_step_size=512and 16.9 / 17.1 GB atprefill_step_size=256(4K / 8K-token prompts, plus 0.64 GB if the DSpark draft is loaded), which fits the default GPU memory limit of a 32 GB Mac: passprefill_step_size=256togenerate, or--prefill-step-size 256tomlx_vlm.server. - DSpark: the bundled draft loads with this build; greedy output follows the model's own except at near-ties (checked: divergences only where the top two tokens were within 0.2 logits). Speeds are under "Speed".
Known limitation of the current server: when the model calls a tool, content can also carry a short sentence of its
reasoning and the tool name.
License and credits
Apache-2.0, as Inkling 2 Mini. Use of the model is also governed by Thinking Machines'
Model Acceptable Use Policy. LICENSE is the Apache-2.0
text and NOTICE describes the conversion. Inkling 2 Mini is by Thinking Machines Lab; please cite their work.
Published by NetProjects (Internet Projects Ltd). Not affiliated with Thinking Machines.
More MLX builds from NetProjects: huggingface.co/netprojects · netprojects.com
- Downloads last month
- 206
4-bit
Model tree for netprojects/Inkling-2-Mini-MLX-4bit
Base model
thinkingmachines/Inkling-2-Mini