You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

Villanova.AI

Villanova2-2B-VL

2.8B parameters · 12.6M training examples · 7.4M images · 21.3B tokens · five European languages · open weight & data

A compact vision-language model built on Villanova2-2B-Base, tuned for reading, understanding and transforming documents in English, Italian, German, Spanish and French. Built in EU, for EU.

2.83B parameters 7.4M unique images MDU-18 58.5 5 languages open weight & data Apache 2.0


Property Value
Model type Vision-language model: Qwen3-VL architecture (Qwen3VLForConditionalGeneration)
Language model Villanova2-2B-Base, 2.411B dense parameters
Vision encoder Qwen3-VL ViT, 0.306B parameters, initialized from Qwen3-VL-4B-Instruct
Total parameters 2,826,553,344 (2.83B)
Inputs / output Text and images / text
Training 4 stages · 12,552,155 examples · 7,360,035 unique images · 21.3B tokens
Openness open weight & data: model weights under Apache 2.0; training data documented source by source, each dataset under its own terms; vision encoder from Qwen3-VL-4B-Instruct (Apache 2.0)
Languages English, Italian, German, Spanish, French
License Apache 2.0

Quick start

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "VillanovaAI/Villanova2-2B-VL"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "path": "invoice.png"},
            {"type": "text", "text": "Estrai numero, data e totale della fattura in formato JSON."},
        ],
    }
]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)

output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(processor.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With vLLM:

vllm serve VillanovaAI/Villanova2-2B-VL --max-model-len 32768 --limit-mm-per-prompt '{"image": 8}'

The tested stacks are transformers 4.57, 5.17, 5.19 and 5.20, and vLLM 0.32. All evaluations below use greedy decoding, the default of the shipped generation_config.json. When no system message is given, the chat template adds a default one ("You are Villanova, a helpful AI assistant built by Villanova.AI.").

The shipped processor encodes every image with 2,304 to 3,328 visual tokens (one token per 32 × 32 pixels after the 2 × 2 merge; smaller images are upscaled), and this is the setting used for all the numbers in this card. Lowering the minimum resolution makes long multi-page prompts cheaper, at some cost on dense pages.

Quantized version

Villanova2-2B-VL-GGUF holds the same weights quantized for llama.cpp, for edge inference; llama-cli handles images natively and downloads the projector automatically:

llama-cli -hf VillanovaAI/Villanova2-2B-VL-GGUF:Q8_0

Intended use

Good fit Not a good fit without extra work
Document transcription to text or Markdown, single and multi-page Safety-critical or high-stakes decisions without human review
Key-value and named-entity extraction from business documents Multi-step visual reasoning and college-level STEM questions (MMMU 30.7)
Page and document summarization and translation in EN · IT · DE · ES · FR Video understanding
Question answering over pages, tables and charts Languages outside the five core ones, where quality is not guaranteed
Fine-tuning into domain-specific document assistants Scene-text reading in natural photos, where other 2B models are clearly stronger (TextVQA 57.5)

The family

Both vision-language models start from the matching Villanova2 base model and see the same multimodal training data in the same three main stages; the 2B adds a short high-resolution stage at the end.


Architecture

Component Value
Language model Villanova2-2B-Base: Qwen3-style dense decoder, 22 layers, hidden 2,560, GQA 20 Q / 4 KV heads
Vision encoder Qwen3-VL ViT: 24 layers, hidden 1,024, 16 heads, patch 16, native dynamic resolution
Projector 2 × 2 spatial merge into the 2,560-dim LM space, plus DeepStack injection of ViT layers 5, 11 and 17 into the first LM layers
Initialization LM from Villanova2-2B-Base; vision encoder and projector from Qwen3-VL-4B-Instruct
Vocabulary 130,822-token Villanova BPE tokenizer · 130,822 embedding rows, tied input/output
Positional encoding Interleaved multimodal RoPE (MRoPE) over text, height and width
Language model parameters 2,411,205,632
Vision encoder parameters 306,242,560
Projector parameters 109,105,152
Total parameters 2,826,553,344

The model uses only native transformers classes: no trust_remote_code, and the same checkpoint loads in transformers and vLLM.


Training

Four stages, one epoch each

Stage What is trained Examples Unique images Tokens Learning rate (LM / projector / ViT)
1 · Alignment Projector only (LM and ViT frozen) 1,046,387 611,969 1.70B — / 1 × 10⁻³ / —
1.5 · Knowledge Full model 3,194,343 3,179,728 6.61B 1 × 10⁻⁵ / 1 × 10⁻⁵ / 2 × 10⁻⁶
2 · Instruction tuning Full model 8,026,641 4,896,956 12.19B 1 × 10⁻⁵ / 1 × 10⁻⁵ / 2 × 10⁻⁶
2.1 · High resolution Full model 284,784 255,488 0.78B 1 × 10⁻⁵ / 1 × 10⁻⁵ / 2 × 10⁻⁶
Total 12,552,155 7,360,035 21.28B

Token counts include the visual tokens. Stages 1.5 and 2 run at a minimum of 1,024 visual tokens per image; stage 2.1 revisits a subset of the instruction data at the release resolution (at least 2,304 visual tokens per image). Stages 1.5 to 2.1 pack examples into sequences of up to 16,384 tokens; the 240 examples longer than that (0.002%) are truncated to that length, and the 9 of them whose cut would fall inside an image are dropped. Unique images are counted across all stages: an image reused in a later stage counts once.


Data

We used synthetic data only from models released under a permissive licence. Where the origin of a dataset was explicit, we removed the proprietary parts, tracing provenance as carefully as we could; where the origin was doubtful, we did not use the data. This distinguishes Villanova2 from many open models, which are post-trained on popular datasets containing answers written by closed models.

Every source is cleared for commercial use. Sources marked (ML) were translated by us from English into Italian, German, Spanish and French; Villanova DU marks the document data we built from FinePDFs pages in the five languages. Where Villanova DU and CommonCanvas needed generated text (questions, answers, summaries, captions), it was produced with Qwen3.5-122B-A10B, self-hosted. Every training example whose image is byte-identical to a page of the MDU test set is removed.

Stage 1 · Alignment · 1.05M examples

Source family Sources Role
Captions PixMo-Cap, brief and detailed (ML) Align the vision features with the language model

Stage 1.5 · Knowledge · 3.19M examples

Source family Sources Role
Captions PixMo-Cap (ML), CommonCanvas captions Dense image descriptions
Document transcription Villanova DU: page to Markdown and to text, single and multi-page Read long documents end to end
Encyclopedic pages Nemotron-VLM-Dataset-v2 wiki pages (10 languages) Rendered pages transcribed to Markdown
Charts PlotQA (ML) Chart question answering

Stage 2 · Instruction tuning · 8.03M examples

Source family Sources Role
General VQA VQAv2, GQA, A-OKVQA, VizWiz (from FineVision), PixMo-AskModelAnything (ML), PixMo-Cap-QA (ML), CommonCanvas VQA, Crossmodal-3600 Natural-image questions and captions
Scene text and OCR TextVQA, DocVQA (from FineVision), TextCaps, SynthDoG-en Reading text in images
Documents Villanova DU: QA, key-value and entity extraction, summarization, translation, classification, layout, anonymization, single and multi-page; Docmatix; FinQA (ML) Document understanding in the five languages
Tables, charts, formulas PubTabNet, SynthTabNet, SynthChartNet, SynthFormulaNet, CoSyn-400K (ML), FigureQA (ML), PlotQA (ML), Nemotron SparseTables Structure recognition and chart QA
Science, math, reasoning AI2D, CLEVR and CLEVR-Math (ML), Nemotron-VLM-Dataset-v2 reasoning subsets (ChartQA, DocVQA, GeomVerse, Mulberry, PlotQA, Visual7W, VisualWebInstruct) Diagrams and step-by-step answers
Grounding Nemotron-VLM-Dataset-v2 Open Images boxes Object localization
Encyclopedic pages Nemotron-VLM-Dataset-v2 wiki pages (10 languages) Long-page transcription
Text only The Instruct set of the Villanova2-2B SFT mixture, including Multi-SFT (Baiamonte et al., 2026), 2.03M conversations in the five languages Keep the language model's chat and reasoning skills

Stage 2.1 · High resolution · 285k examples

A subset of stage 2 drawn from 109 of its 138 sources, revisited at the release resolution so that the model is tuned on the image size it is served at.


Evaluation

Every model in the tables, ours and the external ones, goes through the same pipeline with greedy decoding. For the VLMEvalKit and multilingual tables: VLMEvalKit data, splits and metrics, up to 2,048 new tokens, the LLaVA-1.5 / lmms-eval answer-format instructions, and rule-based answer extraction (exact matching, no LLM judge). Scores are raw. Because the protocol is the same for every model, numbers can differ from those published in other model cards. Openness: weight = open weights only; weight & data = open weights plus documented training data. In the comparison tables, bold marks the best score in each column and underline the best among the open weight & data EU models.

The comparison covers the strongest open models of the same size class and every EU vision-language model that completed the same protocol. EuroVLM-9B is not reported: its run did not produce usable outputs.

General vision-language benchmarks (VLMEvalKit)

Model Openness Origin Params Avg AI2D ChartQA DocVQA MMBench MMMU MMStar MTVQA OCRBench POPE RWQA SEED SQA TextVQA
InternVL3.5-2B weight non-EU 2.3B 71.4 77.6 80.8 88.6 76.2 48.0 57.9 24.0 83.4 88.0 61.4 75.1 90.5 76.2
Qwen3.5-2B weight non-EU 2.3B 71.2 78.2 78.5 93.5 74.1 35.3 55.7 27.6 86.9 89.1 70.3 75.7 82.0 78.7
Ministral-3-3B-Instruct weight EU 3.8B 64.2 74.5 66.6 90.4 66.5 37.3 46.8 26.7 71.9 78.4 58.7 68.3 77.9 70.3
Villanova2-2B-VL weight & data EU 2.8B 60.6 65.1 65.7 85.0 64.2 30.7 43.9 15.0 65.6 86.3 63.7 70.5 75.1 57.5
Velvet2-3B-Vision weight EU 4.3B 58.0 68.7 54.9 66.4 66.2 34.0 41.1 7.9 58.8 88.2 57.8 66.5 76.3 67.8
SmolVLM2-2.2B weight & data non-EU 2.3B 55.4 69.3 26.2 60.7 66.2 37.3 43.9 9.9 60.1 83.1 55.7 70.6 87.6 49.2
gemma-4-E2B weight non-EU 5.1B (2B eff.) 49.8 33.1 42.7 73.7 50.3 7.3 28.3 21.4 72.3 81.9 52.8 65.8 57.8 60.0
TowerVision-2B ‡ weight & data EU 3.0B 48.2 38.5 20.4 53.0 58.0 27.3 35.9 16.7 57.6 88.3 59.5 67.2 51.9 52.3
EuroVLM-1.7B weight & data EU 2.1B 37.4 27.6 19.4 58.0 1.5 22.7 28.5 14.0 59.0 88.3 45.5 33.5 36.8 50.9

‡ TowerVision ends most short answers with a period, which exact matching counts as wrong; ChartQA and TextVQA are the most affected. It runs on transformers 4.57, the only stack that passes it the images.

Villanova2-2B-VL places 4th of 9, ahead of Velvet2-3B-Vision (+2.6), SmolVLM2-2.2B (+5.2), gemma-4-E2B (+10.8), TowerVision-2B (+12.4) and EuroVLM-1.7B (+23.2), and 3.6 points behind Ministral-3-3B, a third larger. Its strongest benchmarks are DocVQA (85.0, about 20 points above Velvet2-3B-Vision and SmolVLM2-2.2B), POPE (86.3) and RealWorldQA (63.7, second only to Qwen3.5-2B). Charts, OCRBench, scene text (TextVQA, MTVQA) and multi-step reasoning (MMMU, MMStar) are where InternVL3.5-2B and Qwen3.5-2B, 10–11 points ahead on the average, lead clearly.

Multilingual benchmarks

Seven VLMEvalKit benchmarks translated into German, Spanish, French and Italian, plus the English originals: 35 tasks. Each prompt, including the answer-format instruction, is in the language of the example. Columns are 5-language averages; MME is the perception + cognition score (0–2,800), rescaled to 0–100 in the overall average.

Model Openness Origin Params Avg AI2D MMBench MME POPE RealWorldQA ScienceQA VQAv2
Qwen3.5-2B weight non-EU 2.3B 73.0 70.5 81.4 1959 85.0 62.0 78.0 64.2
InternVL3.5-2B weight non-EU 2.3B 71.5 69.8 81.5 1935 86.0 51.6 82.2 60.3
Villanova2-2B-VL weight & data EU 2.8B 68.0 61.1 75.8 1719 83.0 54.3 74.3 66.3
Velvet2-3B-Vision weight EU 4.3B 67.2 63.5 74.5 1655 87.2 49.3 72.0 64.5
Ministral-3-3B-Instruct weight EU 3.8B 65.4 65.8 77.1 1713 77.8 52.4 75.0 48.6
gemma-4-E2B weight non-EU 5.1B (2B eff.) 59.9 47.2 70.7 1730 81.3 47.0 65.7 45.8
TowerVision-2B ‡ weight & data EU 3.0B 58.7 50.1 70.8 1302 83.1 45.9 61.9 52.6
SmolVLM2-2.2B weight & data non-EU 2.3B 55.1 50.0 66.6 1149 74.2 41.4 72.1 40.3
EuroVLM-1.7B weight & data EU 2.1B 43.0 25.5 32.1 944 84.8 36.6 37.4 51.0

Per-language profile of Villanova2-2B-VL.

Language Avg AI2D MMBench MME POPE RealWorldQA ScienceQA VQAv2
English 69.8 63.7 76.8 1593 83.4 59.3 75.7 73.1
Italian 67.2 60.0 76.1 1715 81.6 51.9 74.5 65.0
German 66.8 59.6 75.4 1754 82.0 53.3 73.4 61.5
Spanish 67.8 61.0 74.9 1798 84.7 49.3 75.2 65.5
French 68.5 61.2 76.0 1737 83.4 57.5 72.6 66.5

The five languages sit within 3.0 points of each other. English leads mainly on the open-ended VQAv2 answers (73.1, against 61.5–66.5 elsewhere) and on RealWorldQA; on the multiple-choice MMBench the five languages are within 1.9 points.

Document understanding (MDU)

MDU is our multilingual document-understanding benchmark: real-world PDF documents in the five languages, 18 task groups, each scored 0–100 with a reference-based metric (character-level similarity for transcription, micro-F1 on the extracted values for key-value and entity extraction, chrF / chrF++ for summaries and translations, numeric match with tolerance for table and figure questions, token recall for open questions, constraint satisfaction for conditional summaries; no LLM judge). Generation is greedy, with a 65,536-token context. The table groups the 18 task groups into five families; MDU-18 is their plain average.

Model Openness Origin Params MDU-18 Transcr. QA Extr. Summ. Transl.
Villanova2-2B-VL weight & data EU 2.8B 58.5 67.2 53.8 48.1 60.4 58.1
Qwen3.5-2B weight non-EU 2.3B 54.7 63.4 60.7 28.2 55.2 40.5
Ministral-3-3B-Instruct weight EU 3.8B 54.4 51.0 60.6 35.8 61.1 53.0
InternVL3.5-2B weight non-EU 2.3B 49.8 57.2 52.6 26.5 55.9 37.3
gemma-4-E2B weight non-EU 5.1B (2B eff.) 48.1 48.5 52.8 30.9 59.7 32.7
Velvet2-3B-Vision weight EU 4.3B 29.9 33.8 31.8 7.0 41.6 19.6
EuroVLM-1.7B weight & data EU 2.1B 27.2 15.1 40.2 6.6 43.9 14.1
SmolVLM2-2.2B † weight & data non-EU 2.3B 22.7 16.4 31.2 11.7 34.5 6.0
TowerVision-2B †§ weight & data EU 3.0B 19.0 12.1 22.6 8.5 37.3 8.4

† The harness flags these runs as not comparable: too many answers were cut by the generation limit (27.6% for SmolVLM2-2.2B, 56.2% for TowerVision-2B).

§ TowerVision has an 8,192-token context: longer prompts are scored as empty answers, which puts Doc2Markdown, Doc2Text and Document Translation at 0.

All 18 MDU task groups
Task group Villanova2-2B-VL Qwen3.5-2B Ministral-3-3B-Instruct InternVL3.5-2B gemma-4-E2B Velvet2-3B-Vision EuroVLM-1.7B SmolVLM2-2.2B † TowerVision-2B †§
Transcription
Doc2Markdown 59.3 67.8 39.4 50.6 44.0 15.0 7.8 0.0 0.0
Doc2Text 60.9 46.3 47.3 52.9 39.8 13.9 8.3 0.0 0.0
Image2Markdown 87.5 79.6 63.0 71.4 59.2 56.4 9.3 30.0 12.4
Image2Text 91.2 74.8 60.7 70.5 58.6 54.8 15.7 23.4 15.9
Passage Extraction 37.0 48.5 44.7 40.6 40.7 28.6 34.2 28.3 32.3
QA
Question Answering 59.8 62.0 59.8 50.8 48.0 40.2 46.8 41.7 43.8
TableQA 61.3 67.7 71.4 61.7 56.6 31.6 40.9 29.1 40.8
FigureQA 48.5 59.5 59.2 47.7 49.5 36.4 38.9 30.1 38.2
Multi-page QA 53.1 57.2 57.5 53.8 54.8 36.4 42.4 35.2 2.4
Multi-page TableQA 55.0 61.8 63.8 55.6 55.8 20.4 37.1 23.9 6.3
Multi-page figureQA 45.1 56.1 51.9 46.2 52.3 25.9 34.8 27.0 3.8
Extraction
Key-value Extraction 37.0 27.3 35.3 28.5 31.8 8.8 7.3 15.3 9.6
Named-entities extraction 59.2 29.1 36.3 24.4 30.0 5.2 6.0 8.2 7.4
Summarization
Conditional Summarization 80.8 73.8 84.7 76.4 90.5 64.0 68.0 60.1 76.2
Document Summarization 45.7 42.7 47.2 43.5 44.7 23.0 28.0 18.4 1.7
Page Summarization 54.7 49.3 51.4 47.7 43.9 37.8 35.8 25.0 33.9
Translation
Document Translation 49.6 33.1 47.5 31.2 29.1 13.0 11.2 0.0 0.0
Page Translation 66.7 47.9 58.5 43.4 36.4 26.3 16.9 12.0 16.7

Villanova2-2B-VL is a document model first. On MDU it is first of the 9 models, 3.8 points above Qwen3.5-2B and 4.1 above Ministral-3-3B, and 28 points or more above the other small EU models. The lead comes from the tasks where the answer has to be written rather than picked: transcription (+3.8 over the next best), named-entity extraction (59.2 against 36.3 for the runner-up) and translation (+5.1). Summarization is level with Ministral-3-3B; page and table QA trails by about 7 points against Qwen3.5-2B and Ministral-3-3B.

On the multilingual benchmarks it is 3rd of 9 and the best EU model, ahead of Velvet2-3B-Vision (+0.8) and Ministral-3-3B (+2.6), with the highest open-ended VQAv2 average in the table (66.3). On the general benchmarks it sits mid-table, ahead of every EU model of its size except Ministral-3-3B. It is the strongest small model here for multilingual document work, not the best general-purpose 2B vision-language model.


Fine-tuning

  • Start from this checkpoint for document or domain adaptation; LoRA on the language model is enough for most extraction and formatting tasks.
  • Keep the Villanova tokenizer and the shipped chat template: the template adds the BOS token and the <|im_start|> / <|im_end|> turn markers the model was trained with, and generation stops on <|im_end|>.
  • The model was trained in BF16 (DeepSpeed ZeRO-2) on packed sequences of up to 16,384 tokens; BF16 is the recommended default for fine-tuning.
  • Train at the resolution you will serve at. Dense pages benefit from the shipped 2,304-token minimum.

License, citation, acknowledgements

License. Apache 2.0. The vision encoder and projector are initialized from Qwen3-VL-4B-Instruct, released under Apache 2.0; the language model is Villanova2-2B-Base.

Citation. If you use Villanova2-2B-VL, please cite the technical report and the inaugural Villanova paper, Multi-SFT for the multilingual data, and the multilingual vision-language training and evaluation resources:

@inproceedings{simonazzi2026villanova2b,
  author    = {Simonazzi, Stefano and Zeinalipour, Kamyar and Gabburo, Matteo and
               Baiamonte, Daniela and Fano, Elena and Zugarini, Andrea},
  title     = {Villanova-2B: European Open Multilingual and Multimodal Language Models},
  booktitle = {Proceedings of the Twelfth Italian Conference on Computational Linguistics (CLiC-it 2026)},
  editor    = {Basile, Valerio and Croce, Danilo and Passaro, Lucia and Pirrone, Roberto},
  year      = {2026},
  note      = {In press}
}
@techreport{villanova2_2026,
  title       = {Villanova2 --- Technical Report},
  author      = {{Villanova.AI}},
  institution = {Villanova.AI},
  year        = {2026},
  note        = {Coming soon}
}
@misc{baiamonte2026multisft,
  author        = {Baiamonte, Daniela and Zeinalipour, Kamyar and Fano, Elena and
                   Simonazzi, Stefano and Zugarini, Andrea},
  title         = {Multi-SFT: Permissively-Licensed Multilingual Instruction-Following},
  year          = {2026},
  eprint        = {2610.22086},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2610.22086}
}
@article{baiamonte2026multilingual,
  title={Multilingual Training and Evaluation Resources for Vision-Language Models},
  author={Baiamonte, Daniela and Fano, Elena and Gabburo, Matteo and Simonazzi, Stefano and Rigutini, Leonardo and Zugarini, Andrea},
  journal={arXiv preprint arXiv:2604.18347},
  year={2026}
}

Acknowledgements. This work is part of the Villanova project, financed by IPCEI-CIS, Prog. n. SA. 102519 — CUP B29J24000850005.

Villanova2-2B-VL · built in EU, for EU.
Downloads last month
10
Safetensors
Model size
3B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for VillanovaAI/Villanova2-2B-VL

Finetuned
(2)
this model
Quantizations
1 model

Collection including VillanovaAI/Villanova2-2B-VL

Papers for VillanovaAI/Villanova2-2B-VL