Introducing Lumma-Fev-v2
• 7
Sovereign Superintelligence
Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction
Grouped Query Experts: Mixture-of-Experts on GQA Self-Attention
Open research toward Sovereign Superintelligence.
We are a research lab working toward sovereign superintelligence. We believe the most capable models should be efficient enough for every organisation to own and run on its own hardware. So we design new architectures, pre-train our own models from scratch, and release our papers, weights and code openly.