Instructions to use optimum-intel-internal-testing/tiny-random-qwen38-flash-next with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use optimum-intel-internal-testing/tiny-random-qwen38-flash-next with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="optimum-intel-internal-testing/tiny-random-qwen38-flash-next")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("optimum-intel-internal-testing/tiny-random-qwen38-flash-next")
model = AutoModelForMultimodalLM.from_pretrained("optimum-intel-internal-testing/tiny-random-qwen38-flash-next", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use optimum-intel-internal-testing/tiny-random-qwen38-flash-next with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "optimum-intel-internal-testing/tiny-random-qwen38-flash-next"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "optimum-intel-internal-testing/tiny-random-qwen38-flash-next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/optimum-intel-internal-testing/tiny-random-qwen38-flash-next

SGLang

How to use optimum-intel-internal-testing/tiny-random-qwen38-flash-next with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "optimum-intel-internal-testing/tiny-random-qwen38-flash-next" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "optimum-intel-internal-testing/tiny-random-qwen38-flash-next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "optimum-intel-internal-testing/tiny-random-qwen38-flash-next" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "optimum-intel-internal-testing/tiny-random-qwen38-flash-next",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use optimum-intel-internal-testing/tiny-random-qwen38-flash-next with Docker Model Runner:
```
docker model run hf.co/optimum-intel-internal-testing/tiny-random-qwen38-flash-next
```

tiny-random-qwen38-flash-next

A tiny model with random weights and the architecture of Qwen3.8-Flash-Next (qwen4_exp, Qwen4ExpForConditionalGeneration), for testing (optimum-intel / OpenVINO). Its outputs are not meaningful. It needs transformers >= 5.17.

Architecture

Every component type of the real model is present, with the same structure and smaller sizes:

Component	Tiny model	Qwen3.8-Flash-Next
Layers	8 = 2 × (3 Gated DeltaNet + 1 Qwen Sparse Attention)	48 = 12 × (3 + 1)
Hidden size	128	2560
Qwen Sparse Attention (QSA)	4 query / 2 KV heads × 64; indexer 4 heads × 16, blocks of 4 tokens, budget 8 tokens	24 / 2 × 256; indexer 4 × 128, blocks of 4, budget 2048
Gated DeltaNet	2 key / 6 value heads × 8	16 / 48 × 128
MoE	16 experts × 128, top-4, shared expert × 128	512 experts, top-10, shared expert
Hyper-connections	4 streams, low rank 8	4 streams, low rank 320
N-gram PLE (layer 2)	bi- and tri-grams, 2 heads per order, prime vocabularies from 1000, 4 shards	8 heads per order, prime vocabularies from 2·10^7, 128 shards
Multi-Token Prediction head	1 QSA layer (`mtp.*`)	1 QSA layer (`mtp.*`)
Vision encoder	depth 2, hidden 32	depth 27, hidden 1152
Vocabulary, tokenizer, chat template, processors	the real ones

The selection budget is small so that QSA is really sparse on short prompts.

The hidden size and the expert sizes are 128 so that a 4-bit export with the default group size (128) compresses the experts, and the experts can be fused into OpenVINO's MoE operation on GPU. The GPU kernel needs a weight compression group of at least 64, so expert input dimensions below 64 leave the experts uncompressed and unfused. The model has 71M parameters, most of them in the two 248320 × 128 embedding tables.

The checkpoint keeps the layout of the real one:

fused 3D experts.gate_up_proj / experts.down_proj;
the n-gram table split into ngram_embedding.shard_*;
the MTP head under mtp..

transformers does not build the MTP head (_keys_to_ignore_on_load_unexpected = ["^mtp.*"]) and ignores its weights when loading. Its structure follows vLLM (vllm/models/qwen4_exp):

fc_embedding projects the next token's embedding;
a shared fc_hidden projects the main model's pre-final-mixer streams;
the projected embedding is added to every stream;
one QSA decoder layer and the head's hyper_connection_mixer follow, then the shared lm_head.

Initialization

The default initialization (std 0.02) gives nearly uniform logits. Greedy tokens then depend on rounding noise, and comparing runtimes means little. The weights are therefore drawn with more variance:

layers: std 0.09 (about 1 / sqrt(hidden size));
embeddings: std 1;
lm_head: std 2;
MoE routers: std 1.

The same scheme is used for the MTP head.

The model was checked on text, image and video prompts. For both the main model and the MTP head:

outputs are finite, with logits std about 11 and clear greedy top-2 margins (median above 1.3);
32 greedy tokens are all distinct, with no repetition runs;
all experts are used on prompts of 70 tokens and more, 15–16 of 16 per layer on 19-token prompts;
QSA drops 23–93% of the visible tokens.

For the main model, the n-gram PLE contributes 7–21% of the residual-stream norm.

For the MTP head:

its predictions depend on both of its inputs;
a chained second draft step stays healthy.

Downloads last month: 1,059

Safetensors

Model size

72.1M params

Tensor type

I64

F32