Instructions to use mobilint/d1-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mobilint/d1-3B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="mobilint/d1-3B", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("mobilint/d1-3B", trust_remote_code=True, device_map="auto") - Mobilint
How to use mobilint/d1-3B with Mobilint:
# pip install mblt-model-zoo from mblt_model_zoo.vision import MBLT_Engine model = MBLT_Engine( model_cls="d1-3B", model_type="DEFAULT", model_path="", core_mode="global8", ) try: image = model.preprocess("path/to/image.jpg") output = model(image) result = model.postprocess(output) finally: model.dispose() - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use mobilint/d1-3B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "mobilint/d1-3B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mobilint/d1-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/mobilint/d1-3B
- SGLang
How to use mobilint/d1-3B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "mobilint/d1-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mobilint/d1-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "mobilint/d1-3B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mobilint/d1-3B", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use mobilint/d1-3B with Docker Model Runner:
docker model run hf.co/mobilint/d1-3B
About
This repository provides a model compiled and optimized for Mobilint NPU hardware. The model is packaged for deployment on Mobilint's acceleration stack and is intended to be used within that environment.
d1-3B for Mobilint NPUs
LiquidAI/d1-3B (revision 051bcc464b01b9f92942b364d9586b0ef5912432), compiled for the Mobilint
aries-rb NPU. It reads text, JSON and pictures:
the language model runs from d1-3B.mxq and the vision tower and projector from d1-3B-vision.mxq. The prompt is the
checkpoint's own prompt.py, which the model loads from this repository.
import transformers_mblt
from PIL import Image
from transformers import AutoModel
transformers_mblt.register()
model = AutoModel.from_pretrained("mobilint/d1-3B")
print(model.system_one("I was charged twice this month, please refund one of them.", {
"refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
}))
print(model.system_one(None, {
"person": {"type": "noul", "instructions": "Is there a person in the picture?"},
}, images=[Image.open("photo.jpg")]))
model.dispose()
Requirements
- A Mobilint ARIES (
aries-rb) NPU with its driver. pip install "transformers-mblt>=0.8.0": D1 support starts at 0.8.0. It installsmobilint-qb-runtime; this build was tested with qbruntime 1.5.3.AutoModel.from_pretrainedandAutoTokenizer.from_pretrainedload this repository aftertransformers_mblt.register(), or withtrust_remote_code=True(which runsproxy_d1.py).- Core modes: inference scheme
all:global8by default;global4,singlealso run. Passcore_mode=(withtarget_clusters=ortarget_cores=) tofrom_pretrainedto pick one. - KV capacity: 16,382 tokens. A prompt (state and question) of up to 16,382 tokens is accepted; a longer
one raises
ValueError. A prefill call takes longer the deeper it starts in the KV: insingleandglobal4, a prompt past about 8K tokens needs a longer NPU watchdog than qbruntime's default 10 s, such asQBRUNTIME_NPU_TIMEOUT_MS=30000in the environment.global8stays within the default. - Prefill runs in infer calls of at most 2,046 tokens in
global8andglobal4(no call is 0 or 63 tokens modulo 64) and at most 2,048 tokens insingle. - The vision MXQ runs in the same core mode as the text MXQ, one picture per infer call.
- Several questions on one state share its prefill, and a later call on the same state and pictures reuses it. Requests are serialized by a lock, one at a time per model.
Pictures
images takes a list of pictures: PIL.Image objects, as upstream's images= does, or file paths or uint8 arrays.
Each picture is letterboxed on the host into one 512x512 square: scaled with bicubic resampling to fit while
keeping its aspect ratio, centred on black, and normalized to [-1, 1]. The vision MXQ turns it into 256 image
tokens, which open the user turn before the state. Upstream's processor instead splits a large picture into up to ten
512x512 tiles plus a thumbnail; here every picture is one 512x512 input, so fine detail in a large picture is
read at lower resolution.
Quantization
| Item | Setting |
|---|---|
| Weights | 4 bit, value projection 8 bit ("W4V8") |
| Activations | 16 bit by default, including the input embeddings, the first RMSNorm and the first residual add; 130 nodes stay at 8 bit (63 RMSNorm outputs and 67 convolution/projection nodes), and the hidden state leaves the graph through an 8-bit activation (the head) |
| Method | Post-training quantization calibrated on the prompts below |
| Compiler | qbcompiler 1.4.1 |
| Vision MXQ | Vision encoder + projector: 8-bit weights, 16-bit activations except the pixel input and one constant (8 bit), including the patch embedding, the first residual and the projector output; one 512x512 letterboxed picture -> 256 image tokens; calibrated on 320 pictures from the same sources as below. Feature cosine vs fp32: 0.988 |
The compiled text graph ends in an identity Linear head, so it returns the final hidden state rather than vocabulary
logits; the hidden state leaves the graph through that head's activation. The option readout runs on the host: each
option is scored with the bf16 token embedding rows of its tokens (the base model's tied head).
Calibration
657 prompts, 368,368 tokens, at most 7,945 tokens per prompt, rendered with the checkpoint's own prompt.py. 497 text prompts and 160 picture prompts (each picture's 256 image slots filled with the base checkpoint's projector output, computed in fp32 on a GPU). None of the sources is customer data, and no calibration picture is distributed with this repository.
| Source | Share of tokens | Licence |
|---|---|---|
| Tool-routing prompts over an illustrative robot tool catalog, robot-state questions, and passages of the project's own documentation (written for this project) | 33.7% | project material |
| FineWeb / FineWeb-2 passages in 10 languages (ar, de, en, es, fr, ja, ko, pt, ru, zh) | 19.2% short + 13.0% long (4β8K tokens) | ODC-By |
| ToolACE tool catalogs, one routing question each (4β8K tokens) | 12.8% | Apache-2.0 |
| Picture questions: 110 COCO val2017 photos (10 of them with a rendered sign added) and 50 rendered documents, screens and charts (FineWeb text, Noto fonts) | 13.5% | COCO images: Flickr licences per image (CC BY 2.0, CC BY-SA 2.0, CC BY-ND 2.0, no known copyright restrictions); FineWeb ODC-By; Noto SIL OFL 1.1 |
| MMMLU questions in 8 locales (ar, de, es, fr, ja, ko, pt, zh) | 5.4% | MIT |
| JevBench public set, calibration split only | 2.5% | MIT |
Fidelity
Measured on the NPU (text and vision MXQs end to end) against the bf16 base model on a GPU, on questions never used for calibration. KL is the per-question KL divergence of the option distribution (lower is better); agreement is the share of questions where the top option matches bf16. The comparison column is an earlier build of the same text recipe calibrated on text only (and without the 16-bit input embeddings, first RMSNorm and first residual add), run with the same vision MXQ.
| Set | This build | Earlier text-only-calibrated build |
|---|---|---|
| 1,920 picture questions (MMStar EN, K-MMStar KO, COCO, documents/charts, signs), bf16 given the same 512x512 letterboxed picture: mean KL | 0.036 | 0.096 |
| the same, agreement with bf16 | 0.878 | 0.821 |
| the same questions, bf16 with upstream's own preprocessing (tiles): mean KL | 0.063 | 0.116 |
| the same, agreement with bf16 | 0.851 | 0.797 |
| the same, accuracy (bf16: 0.672 letterboxed, 0.673 upstream preprocessing) | 0.647 | 0.634 |
| 840 text questions (JevBench held-out, MMLU in 12 languages, tool routing): mean KL | 0.038 | 0.041 |
| 420 long-context text questions (2Kβ15.5K tokens): mean KL | 0.052 | 0.057 |
The first two picture rows measure the quantization alone; the next two also include the cost of reading each picture as one letterboxed 512x512 input instead of upstream's tiles.
Picture accuracy by set (this build / earlier build / bf16): MMStar 0.573 / 0.530 / 0.588, K-MMStar 0.521 / 0.539 / 0.572, COCO 0.958 / 0.930 / 0.952.
Long context: text accuracy 0.886 against 0.921 for bf16 on the 420 long questions; the 12K-token needle questions lose 3 of 36 against the earlier build, the 15.5K ones improve.
Licence
The base model, LiquidAI/d1-3B, is licensed by Liquid AI, Inc. under the LFM Open License v1.0, a copy of which is in LICENSE. The licence limits commercial use by entities with annual revenue of USD 10 million or more. This repository is a derivative work under the same licence. Modifications to the base model:
- Weights: quantized and compiled for the NPU, the language model into
d1-3B.mxqand the vision tower and projector intod1-3B-vision.mxq. - Token embedding: stored on its own as
model.safetensors(embed_tokens.weight, values unchanged). tokenizer_config.json: 4 keys dropped (backend,is_local,local_files_only,processor_class);tokenizer_classchanged fromTokenizersBackendtoPreTrainedTokenizerFast, so that every Transformers version loads the upstream fast tokenizer class.config.jsonandproxy_d1.pyare new files fortransformers-mblt.
prompt.py, tokenizer.json and LICENSE are byte-identical to the base model revision.
- Downloads last month
- -