About

This repository provides a model compiled and optimized for Mobilint NPU hardware. The model is packaged for deployment on Mobilint's acceleration stack and is intended to be used within that environment.

d1-3B for Mobilint NPUs

LiquidAI/d1-3B (revision 051bcc464b01b9f92942b364d9586b0ef5912432), compiled for the Mobilint aries-rb NPU. It reads text, JSON and pictures: the language model runs from d1-3B.mxq and the vision tower and projector from d1-3B-vision.mxq. The prompt is the checkpoint's own prompt.py, which the model loads from this repository.

import transformers_mblt
from PIL import Image
from transformers import AutoModel

transformers_mblt.register()
model = AutoModel.from_pretrained("mobilint/d1-3B")
print(model.system_one("I was charged twice this month, please refund one of them.", {
    "refund": {"type": "noul", "instructions": "Is the customer asking for a refund?"},
}))
print(model.system_one(None, {
    "person": {"type": "noul", "instructions": "Is there a person in the picture?"},
}, images=[Image.open("photo.jpg")]))
model.dispose()

Requirements

  • A Mobilint ARIES (aries-rb) NPU with its driver.
  • pip install "transformers-mblt>=0.8.0": D1 support starts at 0.8.0. It installs mobilint-qb-runtime; this build was tested with qbruntime 1.5.3.
  • AutoModel.from_pretrained and AutoTokenizer.from_pretrained load this repository after transformers_mblt.register(), or with trust_remote_code=True (which runs proxy_d1.py).
  • Core modes: inference scheme all: global8 by default; global4, single also run. Pass core_mode= (with target_clusters= or target_cores=) to from_pretrained to pick one.
  • KV capacity: 16,382 tokens. A prompt (state and question) of up to 16,382 tokens is accepted; a longer one raises ValueError. A prefill call takes longer the deeper it starts in the KV: in single and global4, a prompt past about 8K tokens needs a longer NPU watchdog than qbruntime's default 10 s, such as QBRUNTIME_NPU_TIMEOUT_MS=30000 in the environment. global8 stays within the default.
  • Prefill runs in infer calls of at most 2,046 tokens in global8 and global4 (no call is 0 or 63 tokens modulo 64) and at most 2,048 tokens in single.
  • The vision MXQ runs in the same core mode as the text MXQ, one picture per infer call.
  • Several questions on one state share its prefill, and a later call on the same state and pictures reuses it. Requests are serialized by a lock, one at a time per model.

Pictures

images takes a list of pictures: PIL.Image objects, as upstream's images= does, or file paths or uint8 arrays. Each picture is letterboxed on the host into one 512x512 square: scaled with bicubic resampling to fit while keeping its aspect ratio, centred on black, and normalized to [-1, 1]. The vision MXQ turns it into 256 image tokens, which open the user turn before the state. Upstream's processor instead splits a large picture into up to ten 512x512 tiles plus a thumbnail; here every picture is one 512x512 input, so fine detail in a large picture is read at lower resolution.

Quantization

Item Setting
Weights 4 bit, value projection 8 bit ("W4V8")
Activations 16 bit by default, including the input embeddings, the first RMSNorm and the first residual add; 130 nodes stay at 8 bit (63 RMSNorm outputs and 67 convolution/projection nodes), and the hidden state leaves the graph through an 8-bit activation (the head)
Method Post-training quantization calibrated on the prompts below
Compiler qbcompiler 1.4.1
Vision MXQ Vision encoder + projector: 8-bit weights, 16-bit activations except the pixel input and one constant (8 bit), including the patch embedding, the first residual and the projector output; one 512x512 letterboxed picture -> 256 image tokens; calibrated on 320 pictures from the same sources as below. Feature cosine vs fp32: 0.988

The compiled text graph ends in an identity Linear head, so it returns the final hidden state rather than vocabulary logits; the hidden state leaves the graph through that head's activation. The option readout runs on the host: each option is scored with the bf16 token embedding rows of its tokens (the base model's tied head).

Calibration

657 prompts, 368,368 tokens, at most 7,945 tokens per prompt, rendered with the checkpoint's own prompt.py. 497 text prompts and 160 picture prompts (each picture's 256 image slots filled with the base checkpoint's projector output, computed in fp32 on a GPU). None of the sources is customer data, and no calibration picture is distributed with this repository.

Source Share of tokens Licence
Tool-routing prompts over an illustrative robot tool catalog, robot-state questions, and passages of the project's own documentation (written for this project) 33.7% project material
FineWeb / FineWeb-2 passages in 10 languages (ar, de, en, es, fr, ja, ko, pt, ru, zh) 19.2% short + 13.0% long (4–8K tokens) ODC-By
ToolACE tool catalogs, one routing question each (4–8K tokens) 12.8% Apache-2.0
Picture questions: 110 COCO val2017 photos (10 of them with a rendered sign added) and 50 rendered documents, screens and charts (FineWeb text, Noto fonts) 13.5% COCO images: Flickr licences per image (CC BY 2.0, CC BY-SA 2.0, CC BY-ND 2.0, no known copyright restrictions); FineWeb ODC-By; Noto SIL OFL 1.1
MMMLU questions in 8 locales (ar, de, es, fr, ja, ko, pt, zh) 5.4% MIT
JevBench public set, calibration split only 2.5% MIT

Fidelity

Measured on the NPU (text and vision MXQs end to end) against the bf16 base model on a GPU, on questions never used for calibration. KL is the per-question KL divergence of the option distribution (lower is better); agreement is the share of questions where the top option matches bf16. The comparison column is an earlier build of the same text recipe calibrated on text only (and without the 16-bit input embeddings, first RMSNorm and first residual add), run with the same vision MXQ.

Set This build Earlier text-only-calibrated build
1,920 picture questions (MMStar EN, K-MMStar KO, COCO, documents/charts, signs), bf16 given the same 512x512 letterboxed picture: mean KL 0.036 0.096
the same, agreement with bf16 0.878 0.821
the same questions, bf16 with upstream's own preprocessing (tiles): mean KL 0.063 0.116
the same, agreement with bf16 0.851 0.797
the same, accuracy (bf16: 0.672 letterboxed, 0.673 upstream preprocessing) 0.647 0.634
840 text questions (JevBench held-out, MMLU in 12 languages, tool routing): mean KL 0.038 0.041
420 long-context text questions (2K–15.5K tokens): mean KL 0.052 0.057

The first two picture rows measure the quantization alone; the next two also include the cost of reading each picture as one letterboxed 512x512 input instead of upstream's tiles.

Picture accuracy by set (this build / earlier build / bf16): MMStar 0.573 / 0.530 / 0.588, K-MMStar 0.521 / 0.539 / 0.572, COCO 0.958 / 0.930 / 0.952.

Long context: text accuracy 0.886 against 0.921 for bf16 on the 420 long questions; the 12K-token needle questions lose 3 of 36 against the earlier build, the 15.5K ones improve.

Licence

The base model, LiquidAI/d1-3B, is licensed by Liquid AI, Inc. under the LFM Open License v1.0, a copy of which is in LICENSE. The licence limits commercial use by entities with annual revenue of USD 10 million or more. This repository is a derivative work under the same licence. Modifications to the base model:

  • Weights: quantized and compiled for the NPU, the language model into d1-3B.mxq and the vision tower and projector into d1-3B-vision.mxq.
  • Token embedding: stored on its own as model.safetensors (embed_tokens.weight, values unchanged).
  • tokenizer_config.json: 4 keys dropped (backend, is_local, local_files_only, processor_class); tokenizer_class changed from TokenizersBackend to PreTrainedTokenizerFast, so that every Transformers version loads the upstream fast tokenizer class.
  • config.json and proxy_d1.py are new files for transformers-mblt.

prompt.py, tokenizer.json and LICENSE are byte-identical to the base model revision.

Downloads last month
-
Safetensors
Model size
0.3B params
Tensor type
BF16
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for mobilint/d1-3B

Finetuned
LiquidAI/d1-3B
Quantized
(27)
this model