Instructions to use OpenInfinity/Index-Translate-2B-MLX-6bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use OpenInfinity/Index-Translate-2B-MLX-6bit with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download OpenInfinity/Index-Translate-2B-MLX-6bit --local-dir Index-Translate-2B-MLX-6bit
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Index-Translate-2B โ MLX 6-bit
English
IndexTeam/Index-Translate-2B converted to MLX for Apple silicon:
- Text only. The official model also reads images. This repo keeps only the language model; the vision encoder is left out.
- 6-bit weights, group size 32. The weights are quantized to 6 bits in groups of 32 (7.0 bits per weight with the scales). Other parameters stay bfloat16.
- Same tokenizer and chat template as the official model.
Files
| File | What | Size |
|---|---|---|
model.safetensors |
6-bit weights | 1,647 MB |
model.safetensors.index.json |
Weight index | |
config.json |
Model configuration, with the quantization settings | |
generation_config.json |
End-of-text token ids | |
tokenizer.json |
The official tokenizer, saved again by the converter | 20 MB |
tokenizer_config.json |
The official file, unchanged | |
chat_template.jinja |
The official chat template | |
LICENSE |
Apache 2.0 | |
SHA256SUMS |
Checksums of every file |
Sizes are in MB = 10โถ bytes.
The converter saves the tokenizer with <think>, </think> and the other special tokens listed in tokenizer.json itself, so loaders that read only tokenizer.json keep them whole. On our 374 test prompts it gives the same token ids as the official tokenizer. tokenizer_config.json is the official file: the converter's copy names a tokenizer class (Qwen3_5Tokenizer) that swift-transformers does not know.
Use
Index-Translate takes one user message and no system message. Use greedy decoding (temperature 0) and turn thinking off, as the official model card does. The prompt names the languages in Chinese:
่ฏทๅฐไปฅไธ{source}ๆๆฌ็ฟป่ฏไธบ{target}๏ผ็ดๆฅ่พๅบ็ฟป่ฏ็ปๆ๏ผไธ่ฆ่ฟ่กไปปไฝ่งฃ้ใ
{text}
For example {source} = ่ฑ่ฏญ (English) and {target} = ไธญๆ (Chinese). Other names: ๆฅ่ฏญ (Japanese), ๅพท่ฏญ (German), ๆณ่ฏญ (French), ่ฅฟ็ญ็่ฏญ (Spanish), ้ฉ่ฏญ (Korean), ไฟ่ฏญ (Russian). See the official model card for glossaries and other constraints.
With the chat template and thinking off, the text the model sees is:
<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n
from mlx_lm import load, generate
model, tokenizer = load("OpenInfinity/Index-Translate-2B-MLX-6bit")
prompt = "่ฏทๅฐไปฅไธ่ฑ่ฏญๆๆฌ็ฟป่ฏไธบไธญๆ๏ผ็ดๆฅ่พๅบ็ฟป่ฏ็ปๆ๏ผไธ่ฆ่ฟ่กไปปไฝ่งฃ้ใ\n\nThe quick brown fox jumps over the lazy dog."
text = tokenizer.apply_chat_template([{"role": "user", "content": prompt}],
add_generation_prompt=True, enable_thinking=False, tokenize=False)
print(generate(model, tokenizer, text, max_tokens=1024))
import Foundation
import MLXLLM
import MLXLMCommon
// modelDirectory: a local copy of this repo. tokenizerLoader: a TokenizerLoader from one of
// mlx-swift-lm's tokenizer integrations (mlx-swift-lm 3 does not bundle one).
// Tested with mlx-swift-lm 3.32.3 and swift-transformers 1.3.4.
let container = try await LLMModelFactory.shared.loadContainer(from: modelDirectory, using: tokenizerLoader)
let session = ChatSession(container,
generateParameters: GenerateParameters(maxTokens: 1024, temperature: 0),
additionalContext: ["enable_thinking": false])
let prompt = "่ฏทๅฐไปฅไธ่ฑ่ฏญๆๆฌ็ฟป่ฏไธบไธญๆ๏ผ็ดๆฅ่พๅบ็ฟป่ฏ็ปๆ๏ผไธ่ฆ่ฟ่กไปปไฝ่งฃ้ใ\n\nThe quick brown fox jumps over the lazy dog."
print(try await session.respond(to: prompt))
How it was converted
From the official weights (revision 9bc0844531) with mlx-lm 0.32.2 (MLX 0.32.4):
mlx_lm.convert --hf-path IndexTeam/Index-Translate-2B --mlx-path Index-Translate-2B-MLX-6bit \
-q --q-bits 6 --q-group-size 32
mlx-lm drops the vision encoder when it converts this model. We chose group size 32 over the default 64: with 64, Chinese output drifted further from the original (see below).
Validation
Test set. 374 text segments taken from 12 real web pages with our browser extension's segmenter. They include links marked as <a1>โฆ</a1> and numbers that must stay unchanged, as the extension sends them.
| Direction | Segments | Pages |
|---|---|---|
| English โ Chinese | 197 | Wikipedia, Hacker News, MDN, Paul Graham, Stack Overflow, GitHub, BBC News, arXiv |
| Chinese โ English | 77 | Chinese Wikipedia, a Chinese blog |
| English โ Japanese | 50 | Wikipedia |
| English โ German | 50 | Wikipedia |
Each segment used the same prompt for every model, built by our app's translation code, with greedy decoding and thinking off. The reference is the official weights converted to MLX without quantization (bfloat16). We compare against the official Q6_K GGUF, run with llama.cpp (build 11540).
Closeness to the original (chrF++ against the bfloat16 reference; 100 = identical):
| Model | Size | All | en โ zh | zh โ en | en โ ja | en โ de | Identical outputs |
|---|---|---|---|---|---|---|---|
| Official GGUF Q6_K | 1,606 MB | 94.2 | 92.1 | 93.1 | 94.6 | 98.2 | 294 / 374 |
| MLX 6-bit, group size 64 | 1,530 MB | 93.7 | 89.7 | 96.3 | 94.1 | 97.2 | 282 / 374 |
| This model (6-bit, group size 32) | 1,647 MB | 94.7 | 91.8 | 96.7 | 94.3 | 97.9 | 285 / 374 |
Checks. Counts of segments with a problem:
| Check | bfloat16 | GGUF Q6_K | This model |
|---|---|---|---|
| Link markers lost or changed (of 140 with links) | 3 | 2 | 4 |
| A number or string that must stay unchanged was changed (of 117) | 13 | 12 | 12 |
The original model misses these too, so most of them come from the model, not from quantization. We also read the 100 segments where this model and the GGUF differ; nearly all differ only in wording, and each side has a few errors the other does not.
Speed (MacBook Pro, M3 Max, 128 GB, macOS 27.2; one segment at a time; this model run with mlx-lm in Python, the GGUF with llama-server):
| GGUF Q6_K | This model | |
|---|---|---|
| All 374 segments | 114.6 s | 94.2 s |
| Generation speed, median | 125 tokens/s | 169 tokens/s |
| Time to first token, median | 42 ms | 35 ms |
| Load time | 4.8 s | 0.9 s |
| Peak memory (process) | 1.8 GB | 2.1 GB |
Many web segments are short, so the time per request weighs more than generation speed: the whole set is 18% faster.
Limitations
- Tested on one machine only: M3 Max with macOS 27.2.
- Text only. For image translation use the official weights.
- Small test set. 374 segments from 12 pages, scored against the original model rather than against human translations.
- No Traditional Chinese. Like the official model, which does not list it.
License and attribution
- License. Apache 2.0, the same as Index-Translate-2B.
- Index-Translate. The model is by IndexTeam, bilibili.
- This repo. We did the MLX conversion, the quantization and the tests. This repo is not affiliated with IndexTeam.
ไธญๆ
ๆฌไปๅบๆฏ IndexTeam/Index-Translate-2B ็ MLX ็ๆฌ๏ผ็จไบ Apple ่ฏ็๏ผ
- ๅชๆๆๅญ้จๅ๏ผๅฎๆนๆจกๅไน่ฝ็ๅพใๆฌไปๅบๅชไฟ็่ฏญ่จๆจกๅ๏ผไธๅซ่ง่ง็ผ็ ๅจใ
- 6 bit ๆ้๏ผๆฏ็ป 32 ไธช๏ผๆ้ๆๆฏ 32 ไธชไธ็ป้ๅไธบ 6 bit๏ผ็ฎไธ็ผฉๆพ็ณปๆฐ็บฆ 7.0 bit/ๆ้๏ผ๏ผๅ ถไฝๅๆฐไฟๆ bfloat16ใ
- ๅ่ฏๅจๅๅฏน่ฏๆจกๆฟไธๅฎๆนๆจกๅ็ธๅใ
ๆไปถ
| ๆไปถ | ๅ ๅฎน | ๅคงๅฐ |
|---|---|---|
model.safetensors |
6 bit ๆ้ | 1,647 MB |
model.safetensors.index.json |
ๆ้็ดขๅผ | |
config.json |
ๆจกๅ้ ็ฝฎ๏ผๅซ้ๅ่ฎพ็ฝฎ | |
generation_config.json |
็ปๆ็ฌฆ id | |
tokenizer.json |
ๅฎๆนๅ่ฏๅจ๏ผ็ฑ่ฝฌๆขๅทฅๅ ท้ๆฐไฟๅญ | 20 MB |
tokenizer_config.json |
ๅฎๆนๅๆไปถ๏ผๆชๆนๅจ | |
chat_template.jinja |
ๅฎๆนๅฏน่ฏๆจกๆฟ | |
LICENSE |
Apache 2.0 | |
SHA256SUMS |
ๆๆๆไปถ็ๆ ก้ชๅ |
ๆฌ้กต็ MB ๆ 10โถ ๅญ่ใ
่ฝฌๆขๅทฅๅ
ทไฟๅญๅ่ฏๅจๆถ๏ผๆ <think>ใ</think> ็ญ็นๆฎๆ ่ฎฐ็ดๆฅๅ่ฟไบ tokenizer.json๏ผๅช่ฏป่ฟไธชๆไปถ็ๅ ่ฝฝๆนๅผไนไธไผๆๅฎไปฌๆๅผใๅจๆไปฌ็ 374 ๆกๆต่ฏๆ็คบ่ฏไธ๏ผๅๅ็ปๆๅๅฎๆนๅ่ฏๅจๅฎๅ
จไธ่ดใtokenizer_config.json ็จ็ๆฏๅฎๆนๅๆไปถ๏ผ่ฝฌๆขๅทฅๅ
ทไฟๅญ็็ๆฌๅ็ๅ่ฏๅจ็ฑปๅ๏ผQwen3_5Tokenizer๏ผswift-transformers ไธ่ฎค่ฏใ
็จๆณ
Index-Translate ๅชๆฅๆถไธๆก็จๆทๆถๆฏ๏ผไธๅ ็ณป็ปๆถๆฏใๆๅฎๆนๆจกๅๅก๏ผ็จ่ดชๅฟ่งฃ็ ๏ผtemperature 0๏ผๅนถๅ ณ้ญๆ่ใๆ็คบ่ฏ้็่ฏญ่จๅ็จไธญๆ๏ผ
่ฏทๅฐไปฅไธ{source}ๆๆฌ็ฟป่ฏไธบ{target}๏ผ็ดๆฅ่พๅบ็ฟป่ฏ็ปๆ๏ผไธ่ฆ่ฟ่กไปปไฝ่งฃ้ใ
{text}
ไพๅฆ {source} = ่ฑ่ฏญ๏ผ{target} = ไธญๆใๅ
ถไป่ฏญ่จๅ๏ผๆฅ่ฏญใๅพท่ฏญใๆณ่ฏญใ่ฅฟ็ญ็่ฏญใ้ฉ่ฏญใไฟ่ฏญใๆฏ่ฏญ่กจ็ญ็บฆๆ็ๅๆณ่งๅฎๆนๆจกๅๅกใ
ๅฅไธๅฏน่ฏๆจกๆฟใๅ ณ้ญๆ่ๅ๏ผๆจกๅๅฎ้ ็ๅฐ็ๆๆฌๆฏ๏ผ
<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n
from mlx_lm import load, generate
model, tokenizer = load("OpenInfinity/Index-Translate-2B-MLX-6bit")
prompt = "่ฏทๅฐไปฅไธ่ฑ่ฏญๆๆฌ็ฟป่ฏไธบไธญๆ๏ผ็ดๆฅ่พๅบ็ฟป่ฏ็ปๆ๏ผไธ่ฆ่ฟ่กไปปไฝ่งฃ้ใ\n\nThe quick brown fox jumps over the lazy dog."
text = tokenizer.apply_chat_template([{"role": "user", "content": prompt}],
add_generation_prompt=True, enable_thinking=False, tokenize=False)
print(generate(model, tokenizer, text, max_tokens=1024))
import Foundation
import MLXLLM
import MLXLMCommon
// modelDirectory๏ผๆฌไปๅบไธ่ฝฝๅฐๆฌๅฐ็็ฎๅฝใtokenizerLoader๏ผmlx-swift-lm ๆไธชๅ่ฏๅจ้ๆๅ
ๆไพ็
// TokenizerLoader๏ผmlx-swift-lm 3 ไธ่ชๅธฆๅ่ฏๅจ๏ผใ
// ๅทฒ็จ mlx-swift-lm 3.32.3 ๅ swift-transformers 1.3.4 ๅฎๆตใ
let container = try await LLMModelFactory.shared.loadContainer(from: modelDirectory, using: tokenizerLoader)
let session = ChatSession(container,
generateParameters: GenerateParameters(maxTokens: 1024, temperature: 0),
additionalContext: ["enable_thinking": false])
let prompt = "่ฏทๅฐไปฅไธ่ฑ่ฏญๆๆฌ็ฟป่ฏไธบไธญๆ๏ผ็ดๆฅ่พๅบ็ฟป่ฏ็ปๆ๏ผไธ่ฆ่ฟ่กไปปไฝ่งฃ้ใ\n\nThe quick brown fox jumps over the lazy dog."
print(try await session.respond(to: prompt))
่ฝฌๆขๆนๆณ
ไปๅฎๆนๆ้๏ผ็ๆฌ 9bc0844531๏ผ็จ mlx-lm 0.32.2๏ผMLX 0.32.4๏ผ่ฝฌๆข๏ผ
mlx_lm.convert --hf-path IndexTeam/Index-Translate-2B --mlx-path Index-Translate-2B-MLX-6bit \
-q --q-bits 6 --q-group-size 32
mlx-lm ่ฝฌๆข่ฟไธชๆจกๅๆถไผๅปๆ่ง่ง็ผ็ ๅจใๆไปฌ้ไบๆฏ็ป 32 ไธชๆ้๏ผ่ไธๆฏ้ป่ฎค็ 64 ไธช๏ผ็จ 64 ๆถ๏ผไธญๆ่พๅบๅๅ็็ๅทฎ่ทๆดๅคง๏ผ่งไธๆ๏ผใ
้ช่ฏ
ๆต่ฏ้๏ผ็จๆไปฌๆต่งๅจๆฉๅฑ็ๅๆฎต้ป่พ๏ผไป 12 ไธช็ๅฎ็ฝ้กตๅไบ 374 ๆฎตๆๅญใๅ
ถไธญๅธฆๆ <a1>โฆ</a1> ๅฝขๅผ็้พๆฅๆ ่ฎฐ๏ผไปฅๅๅฟ
้กปๅๆ ทไฟ็็ๆฐๅญ๏ผๅๆฉๅฑๅฎ้
ๅๅบ็ไธๆ ทใ
| ๆนๅ | ๆฎตๆฐ | ็ฝ้กต |
|---|---|---|
| ่ฑ่ฏไธญ | 197 | ็ปดๅบ็พ็งใHacker NewsใMDNใPaul GrahamใStack OverflowใGitHubใBBC ๆฐ้ปใarXiv |
| ไธญ่ฏ่ฑ | 77 | ไธญๆ็ปดๅบ็พ็งใไธไธชไธญๆๅๅฎข |
| ่ฑ่ฏๆฅ | 50 | ็ปดๅบ็พ็ง |
| ่ฑ่ฏๅพท | 50 | ็ปดๅบ็พ็ง |
ๆฏๆฎตๅฏนๆๆๆจกๅ็จๅไธๆกๆ็คบ่ฏ๏ผ็ฑๆไปฌๅบ็จ็็ฟป่ฏไปฃ็ ็ๆ๏ผ๏ผ่ดชๅฟ่งฃ็ ใๅ ณ้ญๆ่ใๅ็ งๆฏๅฎๆนๆ้ไธ้ๅ็ดๆฅ่ฝฌๆ MLX๏ผbfloat16๏ผใๅฏนๆฏๅฏน่ฑกๆฏๅฎๆน็ Q6_K GGUF๏ผ็จ llama.cpp๏ผbuild 11540๏ผ่ฟ่กใ
ๅๅ็็ๆฅ่ฟ็จๅบฆ๏ผไปฅ bfloat16 ๅ็ ง่ฎก็ฎ chrF++๏ผ100 ่กจ็คบๅฎๅ จ็ธๅ๏ผ๏ผ
| ๆจกๅ | ๅคงๅฐ | ๅ จ้จ | ่ฑ่ฏไธญ | ไธญ่ฏ่ฑ | ่ฑ่ฏๆฅ | ่ฑ่ฏๅพท | ไธๅ็ งๅฎๅ จ็ธๅ |
|---|---|---|---|---|---|---|---|
| ๅฎๆน GGUF Q6_K | 1,606 MB | 94.2 | 92.1 | 93.1 | 94.6 | 98.2 | 294 / 374 |
| MLX 6 bit๏ผๆฏ็ป 64 ไธช | 1,530 MB | 93.7 | 89.7 | 96.3 | 94.1 | 97.2 | 282 / 374 |
| ๆฌๆจกๅ๏ผ6 bit๏ผๆฏ็ป 32 ไธช๏ผ | 1,647 MB | 94.7 | 91.8 | 96.7 | 94.3 | 97.9 | 285 / 374 |
ๆฃๆฅ๏ผๅบ็ฐ้ฎ้ข็ๆฎตๆฐ๏ผ
| ๆฃๆฅ้กน | bfloat16 | GGUF Q6_K | ๆฌๆจกๅ |
|---|---|---|---|
| ้พๆฅๆ ่ฎฐไธขๅคฑๆๆนๅจ๏ผๅ ฑ 140 ๆฎตๅธฆ้พๆฅ๏ผ | 3 | 2 | 4 |
| ๅฟ ้กปๅๆ ทไฟ็็ๆฐๅญๆๅญ็ฌฆไธฒ่ขซๆนๅจ๏ผๅ ฑ 117 ๆฎต๏ผ | 13 | 12 | 12 |
ๅ็ๆจกๅไนไผๅบ่ฟไบ้ฎ้ข๏ผๆไปฅๅคง้จๅๆฅ่ชๆจกๅๆฌ่บซ๏ผไธๆฏ้ๅ้ ๆ็ใๆไปฌ่ฟ้ๆก็ไบๆฌๆจกๅๅ GGUF ็ปๆไธๅ็ 100 ๆฎต๏ผๅ ไน้ฝๅชๆฏๆช่พไธๅ๏ผไธค่พนๅๆๅฐๆฐๅฏนๆนๆฒกๆ็้่ฏฏใ
้ๅบฆ๏ผMacBook Pro๏ผM3 Max๏ผ128 GB๏ผmacOS 27.2๏ผ้ๆฎต็ฟป่ฏ๏ผๆฌๆจกๅ็จ Python ็ mlx-lm ่ฟ่ก๏ผGGUF ็จ llama-server ่ฟ่ก๏ผ๏ผ
| GGUF Q6_K | ๆฌๆจกๅ | |
|---|---|---|
| 374 ๆฎตๆป็จๆถ | 114.6 ็ง | 94.2 ็ง |
| ็ๆ้ๅบฆ๏ผไธญไฝๆฐ๏ผ | 125 token/็ง | 169 token/็ง |
| ้ฆไธช token ็จๆถ๏ผไธญไฝๆฐ๏ผ | 42 ๆฏซ็ง | 35 ๆฏซ็ง |
| ๅ ่ฝฝๆถ้ด | 4.8 ็ง | 0.9 ็ง |
| ๅ ๅญๅณฐๅผ๏ผ่ฟ็จ๏ผ | 1.8 GB | 2.1 GB |
็ฝ้กต้ๅพๅคๆฏ็ญๅฅ๏ผๆฏๆฌก่ฏทๆฑ็ๅบๅฎๅผ้ๆฏ็ๆ้ๅบฆๅฝฑๅๆดๅคง๏ผๆไปฅๆดๅฅๅชๅฟซ 18%ใ
ๅฑ้
- ๅชๅจไธๅฐๆบๅจไธๆต่ฟ๏ผM3 Max๏ผmacOS 27.2ใ
- ๅช่ฝ็ฟป่ฏๆๅญ๏ผ่ฆ็ฟป่ฏๅพ็่ฏท็จๅฎๆนๆ้ใ
- ๆต่ฏ้ๅพๅฐ๏ผ12 ไธช็ฝ้กตๅ ฑ 374 ๆฎต๏ผไปฅๅ็ๆจกๅไธบๅ็ ง๏ผๆฒกๆ็จไบบๅทฅ่ฏๆ่ฏๅใ
- ไธๆฏๆ็นไฝไธญๆ๏ผๅๅฎๆนๆจกๅไธๆ ท๏ผๅฎๆน่ฏญ่จๅ่กจ้ๆฒกๆ็นไฝไธญๆใ
่ฎธๅฏ่ฏๅๆฅๆบ
- ่ฎธๅฏ่ฏ๏ผApache 2.0๏ผๅ Index-Translate-2B ็ธๅใ
- Index-Translate๏ผๆจกๅ็ฑๅๅฉๅๅฉ IndexTeam ๅๅธใ
- ๆฌไปๅบ๏ผๆไปฌๅไบ MLX ่ฝฌๆขใ้ๅๅๆต่ฏใๆฌไปๅบไธ IndexTeam ๆฒกๆๅ ณ่ใ
- Downloads last month
- 99
6-bit
Model tree for OpenInfinity/Index-Translate-2B-MLX-6bit
Base model
IndexTeam/Index-Translate-2B