Some models (like https://huggingface.co/embedl/sam3) have more benchmarks in the model page - will push for updating the Benchmarks page with those results.
Jonna Matthiesen
JonnaMat
AI & ML interests
None yet
Recent Activity
Organizations
replied to their post about 11 hours ago
replied to their post about 11 hours ago
Yes, accuracy numbers are a bit thin! For FlashHead you are correct. We have written a paper on FlashHead (see https://arxiv.org/abs/2603.14591) where we show that it is consistently accuracy-preserving: typically within ~99.9% top-1 containment.
For the GenAI models on Jetson: We are actually using the TensorRT vLLM Docker image, so lm-eval has a path and is not blocked by the TRT engine. For example, for Qwen3.6 we have internal accuracy evaluations but not yet pushed.
For the more classical DL models, the benchmark suite mostly compares to quantized variants where we do need to push for more on-device accuracy evaluations.
The notion of picking the largest model that fits the latency (and memory) budget is quite common and that's one focus of our GenAI models . On the classical DL side it is a bit different, edge deployment use-cases are often around making an existing, well performing model faster.
posted an update 1 day ago
Post
837
🐆💨 How small can an LLM go and still run on Jetson? We benchmarked + released the whole suite on NVIDIA - LLMs, VLMs, vision, speech, embeddings: embedl/Edge-Inference-Benchmarks
What use cases do you have for those models?
What use cases do you have for those models?
- 4 replies
reacted to HannesVonEssen's post with 🔥 3 months ago
Post
283
📣 I made a visualizer for Hugging Face models: https://hfviewer.com
✨ Simply paste a Hugging Face URL to get an interactive visualization of the architecture!
🔗 The recent Qwen3.6-27B model as an example: https://hfviewer.com/Qwen/Qwen3.6-27B
Feel free to try it out and give me feedback on how it can be improved! ❤️
✨ Simply paste a Hugging Face URL to get an interactive visualization of the architecture!
🔗 The recent Qwen3.6-27B model as an example: https://hfviewer.com/Qwen/Qwen3.6-27B
Feel free to try it out and give me feedback on how it can be improved! ❤️
- 1 reply
reacted to HannesVonEssen's post with 🔥❤️ 3 months ago
Post
11678
📣 Hugging Face Visualizer, now as Chrome extension!
https://hfviewer.com
✨ After installing, Hugging Face model pages will have an architecture visualization on the model page itself!
🔗 Link:
https://chromewebstore.google.com/detail/hugging-face-viewer/mmadlggmpkpiockpjfepaohcllbnakej
Thanks for all the nice feedback so far! ❤️
https://hfviewer.com
✨ After installing, Hugging Face model pages will have an architecture visualization on the model page itself!
🔗 Link:
https://chromewebstore.google.com/detail/hugging-face-viewer/mmadlggmpkpiockpjfepaohcllbnakej
Thanks for all the nice feedback so far! ❤️
- 5 replies
posted an update 3 months ago
Post
150
⚡ Qwen3.5, up to 1.4× faster. Same quality. Less latency.
We applied FlashHead to the Qwen3.5 family: Novel drop-in replacement of the LM head with measurably lower latency on edge hardware. Benchmarks and models below.
📊 embedl/Edge-Inference-Benchmarks
🤗 https://huggingface.co/collections/embedl/qwen35
We applied FlashHead to the Qwen3.5 family: Novel drop-in replacement of the LM head with measurably lower latency on edge hardware. Benchmarks and models below.
📊 embedl/Edge-Inference-Benchmarks
🤗 https://huggingface.co/collections/embedl/qwen35
posted an update 4 months ago
Post
120
⚡ FlashHead benchmarks for Llama 3.2, Gemma 3, and Qwen3 are now on embedl/Edge-Inference-Benchmarks !
These are some of the models used in the FlashHead paper - now easier to explore and compare interactively.
🚀 Jetson AGX Thor (tok/s, batch=1):
- Llama-3.2-1B: 77 → 285 (FlashHead+W4A16, 3.7x)
- Llama-3.2-3B: 34 → 112 (3.3x)
- Gemma-3-1B: 79 → 153 (1.9x)
- Qwen3-1.7B: 49 → 189 (3.8x)
- Qwen3-0.6B: 140 → 177 (1.3x)
✅ Accuracy matches baseline on MMLU-Pro, IFEval, BBH, TruthfulQA, GSM8K.
These are some of the models used in the FlashHead paper - now easier to explore and compare interactively.
🚀 Jetson AGX Thor (tok/s, batch=1):
- Llama-3.2-1B: 77 → 285 (FlashHead+W4A16, 3.7x)
- Llama-3.2-3B: 34 → 112 (3.3x)
- Gemma-3-1B: 79 → 153 (1.9x)
- Qwen3-1.7B: 49 → 189 (3.8x)
- Qwen3-0.6B: 140 → 177 (1.3x)
✅ Accuracy matches baseline on MMLU-Pro, IFEval, BBH, TruthfulQA, GSM8K.
replied to their post 4 months ago
Any plans on releasing flashhead for qwen3.5 models?
Yes! FlashHead-enabled Qwen3.5 models are coming soon. We are currently finalizing accuracy and latency evaluations.
posted an update 4 months ago
Post
1267
⚡ FlashHead: Fast LM Head Inference - Now a Simple vLLM Plugin
flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just:
✨ The plugin activates automatically via vLLM's
🧩 Supported models (full collection):
Qwen Qwen3, 
meta-llama Llama3, 
google Gemma3, 
nvidia Cosmos-Reason2 - BF16 and W4A16 variants.
https://huggingface.co/collections/embedl/flashhead
📊 embedl/Edge-Inference-Benchmarks
🔧 Benchmark it yourself:
FlashHead shines at low batch sizes; the typical real-time / on-device use case. 🚀
flash-head replaces the dense LM head with a two-stage retrieval pipeline - up to 2x inference speedup, training-free. Previously required custom Docker images; now it's just:
pip install flash-head
vllm serve embedl/Qwen3-1.7B-FlashHead-W4A16✨ The plugin activates automatically via vLLM's
vllm.general_plugins entry point. No source patches, no custom imports.🧩 Supported models (full collection):
https://huggingface.co/collections/embedl/flashhead
📊 embedl/Edge-Inference-Benchmarks
🔧 Benchmark it yourself:
vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1
# Baseline comparison
FLASHHEAD_ENABLED=0 vllm bench latency --model embedl/Qwen3-1.7B-FlashHead-W4A16 --batch-size 1FlashHead shines at low batch sizes; the typical real-time / on-device use case. 🚀
- 2 replies
replied to their post 4 months ago
Check it out here: https://github.com/JonnaMat/huggingface-slack-app
posted an update 4 months ago
Post
121
🕵 Want to know the second a competitor drops a new model... in Slack? 🔔
I built a Slack integration that lets you "watch" any organization or model on the Hub.
🚀 What it does:
- Pings your team when a watched org publishes a new model/dataset.
- Tracks real-time engagement (Likes, Downloads, Followers) on your own releases.
- Automated milestones so you can stop manually checking metrics.
Built with Python, uv - we all love the dopamine hit seeing the downloads climb. 🥂
I built a Slack integration that lets you "watch" any organization or model on the Hub.
🚀 What it does:
- Pings your team when a watched org publishes a new model/dataset.
- Tracks real-time engagement (Likes, Downloads, Followers) on your own releases.
- Automated milestones so you can stop manually checking metrics.
Built with Python, uv - we all love the dopamine hit seeing the downloads climb. 🥂
- 1 reply
posted an update 4 months ago
Post
171
HuggingFace x Slack
We all love watching those download numbers climb on the Hub or celebrating when we release a new model. I built a Slack app that turns those milestones into team celebrations. 🥂
👀 The "Secret" Feature: It’s not just for your own wins. You can use it to keep a pulse on what other organizations are up to. Track their new model drops or download spikes... Sometimes even before the official announcements. 🕵️♂️
🔗 Check out the repo and give it a star if you like it: https://github.com/JonnaMat/huggingface-slack-app
#OpenSource #InferenceAPI #SlackBot #CompetitiveIntelligence #MachineLearning #HuggingFaceAPI
We all love watching those download numbers climb on the Hub or celebrating when we release a new model. I built a Slack app that turns those milestones into team celebrations. 🥂
👀 The "Secret" Feature: It’s not just for your own wins. You can use it to keep a pulse on what other organizations are up to. Track their new model drops or download spikes... Sometimes even before the official announcements. 🕵️♂️
🔗 Check out the repo and give it a star if you like it: https://github.com/JonnaMat/huggingface-slack-app
#OpenSource #InferenceAPI #SlackBot #CompetitiveIntelligence #MachineLearning #HuggingFaceAPI
reacted to AINovice2005's post with 👍 5 months ago
Post
389
Pro tip: If you are finetuning any model with tensorboard logs enabled, be sure to upload them to HF Hub as event artifacts, they can be viewed instantly. 🚀
I previously remembered this done in the notus model release: argilla/notus-7b-v1
Examples:
AINovice2005/ModernBERT-base-lora-cicflow-1m-r8
AINovice2005/ModernBERT-base-lora-cicflow-1m-r4
AINovice2005/ModernBERT-base-lora-cicflow-1m-r16
cc: @davidberenstein1957
I previously remembered this done in the notus model release: argilla/notus-7b-v1
Examples:
AINovice2005/ModernBERT-base-lora-cicflow-1m-r8
AINovice2005/ModernBERT-base-lora-cicflow-1m-r4
AINovice2005/ModernBERT-base-lora-cicflow-1m-r16
cc: @davidberenstein1957
replied to their post 5 months ago
🤓 Want to learn more about FlashHead? Check out this blog post: https://huggingface.co/blog/JonnaMat/flashhead
posted an update 5 months ago
Post
5949
🚀 FlashHead: Efficient Drop-In Replacement for the Classification Head in Language Model Inference
🔎 Check out our latest FlashHead-enabled model: embedl/Cosmos-Reason2-2B-W4A16-Edge2-FlashHead
🧩 Seamless integration with vllm:
🔎 Check out our latest FlashHead-enabled model: embedl/Cosmos-Reason2-2B-W4A16-Edge2-FlashHead
🧩 Seamless integration with vllm:
docker run --rm -it \
--network host \
--shm-size=8g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
--runtime=nvidia \
--name=vllm-serve \
-e HF_TOKEN=hf_*** \
-e HF_HOME=/root/.cache/huggingface \
embedl/vllm:latest-jetson-orin-flashhead \
vllm serve "embedl/Cosmos-Reason2-2B-W4A16-Edge2-FlashHead" \
--max-model-len 8192 \
--gpu-memory-utilization 0.75 \
--max-num-seqs 2 \
--trust-remote-code- 1 reply
replied to their post 5 months ago
Additional Jetson Orin Nano Super benchmarks will be added continuously.
posted an update 5 months ago
Post
1091
Qwen3.5 on-device benchmarks on the Nvidia Jetson lineup are now live 🚀
We've added the latest Qwen3.5 models (0
8B - 9B) to our on-device inference benchmarks (Nvidia Jetson Orin Nano Super, AGX Orin, AGX Thor).
👉 Explore TPS, TTFT, E2E latency, and TPOT. Measured on real hardware: embedl/Edge-Inference-Benchmarks
🌟 Stay tuned for additional benchmarks and Embedl-optimized models: Enabling models run faster and on less expensive hardware.
If you're working on edge LLM deployment, we'd love to discuss your use case.
We've added the latest Qwen3.5 models (0
8B - 9B) to our on-device inference benchmarks (Nvidia Jetson Orin Nano Super, AGX Orin, AGX Thor).
👉 Explore TPS, TTFT, E2E latency, and TPOT. Measured on real hardware: embedl/Edge-Inference-Benchmarks
🌟 Stay tuned for additional benchmarks and Embedl-optimized models: Enabling models run faster and on less expensive hardware.
If you're working on edge LLM deployment, we'd love to discuss your use case.
- 1 reply
posted an update 5 months ago
Post
1621
🤯 Edge-Grade Vision Reasoning. Now Practically Lossless. 🤯
Introducing
👉 embedl/Cosmos-Reason2-2B-W4A16-Edge2
Optimized for Jetson Orin Nano Super and AGX Orin nvidia .
🚄 Try it out on Jetson (image+video+text):
🤓 What is Edge2? Most weights → INT4 | Activations → FP16 | Select sensitive layers → kept in FP16.
Edge2 preserves precision where it matters most; while keeping the model small and fast enough for edge GPUs. 😎
Introducing
👉 embedl/Cosmos-Reason2-2B-W4A16-Edge2
Optimized for Jetson Orin Nano Super and AGX Orin
🚄 Try it out on Jetson (image+video+text):
docker run --rm -it \
--network host \
--shm-size=8g \
--ulimit memlock=-1 \
--ulimit stack=67108864 \
--runtime=nvidia \
--name=vllm-serve \
-e HF_TOKEN=hf_*** \
-e HF_HOME=/root/.cache/huggingface \
ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
vllm serve "embedl/Cosmos-Reason2-2B-W4A16-Edge2" \
--max-model-len 8192 \
--gpu-memory-utilization 0.75 \
--max-num-seqs 2🤓 What is Edge2? Most weights → INT4 | Activations → FP16 | Select sensitive layers → kept in FP16.
Edge2 preserves precision where it matters most; while keeping the model small and fast enough for edge GPUs. 😎
reacted to HannesVonEssen's post with 🚀 5 months ago
Post
1374
Cosmos-Reason2-2B on Nano Super
Hi! Today, me and my team is releasing a version of Cosmos-Reason2-2B that is quantized so that it fits on the NVIDIA Jetson Orin Nano Super.
We managed to find a mixed precision configuration such that it maintains virtually the same accuracy as the unquantized model while being able to run really efficiently on the Nano Super and other edge devices :)
embedl/Cosmos-Reason2-2B-W4A16-Edge2
Hi! Today, me and my team is releasing a version of Cosmos-Reason2-2B that is quantized so that it fits on the NVIDIA Jetson Orin Nano Super.
We managed to find a mixed precision configuration such that it maintains virtually the same accuracy as the unquantized model while being able to run really efficiently on the Nano Super and other edge devices :)
embedl/Cosmos-Reason2-2B-W4A16-Edge2