deepseek-ai/DeepSeek-V4.1-Flash-1017
Image-Text-to-Text • 763B • Updated • 17.9k • • 1.57k
Jack Voide
Mindweller
Quazim0t0's profile picture
webbrain-one-954747's profile picture
Arunsajeevan199303's profile picture·
AI & ML interests
None yet
Recent Activity
liked a model about 2 hours ago
deepseek-ai/DeepSeek-V4.1-Flash-1017 liked a model 3 days ago
prism-ml/Ternary-Bonsai-2-9B-gguf liked a model 5 days ago
unsloth/Qwen-Image-2.1-Turbo-FP8Organizations
liked a
model about 2 hours ago
liked a
model 3 days ago
liked 2
models 5 days ago
reacted to SeaWolf-AI's post with 👍 5 days ago
Post
2,741
🚀 We pretrained a language model from scratch on an inference-only NPU, with real backpropagation.
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
🤗 Model: FINAL-Bench/VKORE-46M-NPU 📝 Technical write-up: https://huggingface.co/blog/FINAL-Bench/backpropagation-on-an-inference-only-npu-pretraini
"NPUs are inference-only, you can't train on them." We built a counterexample: VKORE-46M-NPU, trained from scratch on a first-gen (2025) inference NPU, 32 chips, no pretrained weights. A 46M decoder-only model (RoPE · SwiGLU · RMSNorm · GQA), char-level, TinyStories.
📏 Why trust it: a falling-loss curve proves nothing. So we trained the same model under identical init, data, and schedule on the NPU and on a GPU reference. · Loss trajectories correlate at 0.99 · Throughput converged to a comparable level The NPU backward pass and optimizer updates do numerically the same work as the GPU, not an approximation that merely trends down.
🧩 One accelerator now covers pretraining · LoRA · inference · quantization. We call this layer VKORE: "CUDA for NPUs."
⚖️ Public: weights, model card, NPU-vs-GPU comparison. Private (patent-pending, paper to follow): the core training technique. 46M char-level is toy scale on purpose: it proves the training path, not model quality. Larger models and real data are next.
Feedback from folks working on custom autograd backends and low-precision training kernels on non-GPU accelerators is very welcome. 🙌
- 10 replies
reacted to Avifenesh's post with 👍 5 days ago
Post
2,161
I added a small gate to the Gated DeltaNet layers of Qwen3.5 and trained it with GRPO under a length penalty.
On Qwen3.5-4B it reasons 48% shorter than LoRA on MATH-500, and anytime accuracy goes from 0.786 to 0.852. A LoRA with about the same parameter count barely moves.
The gate sits on beta, the per-token write strength. One matrix per layer, zero init, so it starts as a no-op.
It started as a negative eigenvalue experiment. The negative range turned out not to matter. The gate did.
Write-up: https://huggingface.co/blog/Avifenesh/beta-gate-qwen35
Adapters: Avifenesh/gate-qwen35-6accb63081800b2f2d583b80
Code: https://github.com/avifenesh/gdn-beta-gatehttps://huggingface.co/collections/Avifenesh/gate-qwen35
On Qwen3.5-4B it reasons 48% shorter than LoRA on MATH-500, and anytime accuracy goes from 0.786 to 0.852. A LoRA with about the same parameter count barely moves.
The gate sits on beta, the per-token write strength. One matrix per layer, zero init, so it starts as a no-op.
It started as a negative eigenvalue experiment. The negative range turned out not to matter. The gate did.
Write-up: https://huggingface.co/blog/Avifenesh/beta-gate-qwen35
Adapters: Avifenesh/gate-qwen35-6accb63081800b2f2d583b80
Code: https://github.com/avifenesh/gdn-beta-gatehttps://huggingface.co/collections/Avifenesh/gate-qwen35
reacted to tardellirs's post with 👍 6 days ago
Post
5,003
Crivo-150M is now on the Pareto frontier of the Decision Index 🎯
Scored by the maintainers on the private tests: Full 22.21, the best of all submissions at or below 150M and higher than every model below 0.75B, including models up to 4× its size. Median latency on the board's GPU: 13 ms.
It is a 149M typed-decision encoder (Ettin / ModernBERT): a state plus choice, noul or score questions in, a probability for every option out, in one forward pass. Trained on open data only.
Next: tasks from new domains, the weak spot of every small encoder on the board.
tardellirs/crivo-150m
multimodalart/jev-decision-index
Thanks to @multimodalart for the Decision Index and its open kit, @orionweller and the Ettin team, @tomaarsen for the Ettin rerankers, @hotchpotch for the Bekko System One data and @sileod for tasksource.
Scored by the maintainers on the private tests: Full 22.21, the best of all submissions at or below 150M and higher than every model below 0.75B, including models up to 4× its size. Median latency on the board's GPU: 13 ms.
It is a 149M typed-decision encoder (Ettin / ModernBERT): a state plus choice, noul or score questions in, a probability for every option out, in one forward pass. Trained on open data only.
Next: tasks from new domains, the weak spot of every small encoder on the board.
tardellirs/crivo-150m
multimodalart/jev-decision-index
Thanks to @multimodalart for the Decision Index and its open kit, @orionweller and the Ettin team, @tomaarsen for the Ettin rerankers, @hotchpotch for the Bekko System One data and @sileod for tasksource.
liked 2
models 6 days ago
liked a
Space 7 days ago
liked a
model 14 days ago
liked a
model 16 days ago
liked a
model 18 days ago
reacted to pollix's post with 👍👍 18 days ago
Post
6,452
stuntd 0.1.2 is out 🎉
stuntd sits in front of your LLM, learns its typed decisions and answers the confident ones locally with a small head on the Laya encoder by @convaiinnovations . About 20ms on GPU and 60ms on CPU, and anything it isn't sure about still goes to the big model.
New in 0.1.2:
- decisions with several fields, like category + urgency + needs_human in one call, answered locally only when every field is sure
- the Anthropic Messages API learns too, not only OpenAI
- auto_retrain: the daemon retrains a site in the background once enough new traffic comes in, so collect, train, shadow and live run on their own
- serve --lazy loads the checkpoint on the first request
Try it in the browser: pollix/stuntd
Code: https://github.com/bladedevoff/stuntd
stuntd sits in front of your LLM, learns its typed decisions and answers the confident ones locally with a small head on the Laya encoder by @convaiinnovations . About 20ms on GPU and 60ms on CPU, and anything it isn't sure about still goes to the big model.
New in 0.1.2:
- decisions with several fields, like category + urgency + needs_human in one call, answered locally only when every field is sure
- the Anthropic Messages API learns too, not only OpenAI
- auto_retrain: the daemon retrains a site in the background once enough new traffic comes in, so collect, train, shadow and live run on their own
- serve --lazy loads the checkpoint on the first request
Try it in the browser: pollix/stuntd
Code: https://github.com/bladedevoff/stuntd
pip install -U stuntd
- 1 reply
liked 5
models 22 days ago
Akahsizrr/Cyber-Prime-1.1-2.6B
Text Generation • 3B • Updated • 1.08k • 12
onecat-ai/OneCAT-3B
9B • Updated • 94 • 15
onecat-ai/LDF-VFI
Image-to-Video • 1B • Updated • 10
FINAL-Bench/Darwin-27B-RSI
Text Generation • 27B • Updated • 1.52k • 35
FINAL-Bench/POCKET-Zimage-CPU
Text-to-Image • 6B • Updated • 2.08k • 52