Sol Labs is an independent research project. We train small language models and publish the checkpoints here. There's also SolPix, our text-to-image model.

ChatSLM

ChatSLM is a chatbot-style application where you can try community AI models, use them through a keyless API, and compare models head-to-head in the Arena. You can try it out here: https://chatslm.pages.dev

Downloads

Nano and Lite Base use PyTorch. Milkshake and Lassi run with MLX on Apple Silicon. All four are base models for completing text. The cards include loading examples, parameter counts, context lengths, and training exposures.

Current experiments

Lite Base has ten transformer blocks, with four of them used twice. That's fourteen block applications from the same stored weights. We also test EngramLite and tensorized n-gram memory, grouped-query attention, and compact tokenizers.

Training these small models takes large token budgets. Our mixtures include educational web text, math, procedural writing, and code, with source weights changing during a run. On the optimizer side, we're testing SimO and SimOW, gated and momentum variants, and other update rules. We compare held-out loss and downstream scores alongside training loss.

SolPix uses a compact flow transformer over compressed image latents. We train the generator and keep the text encoder and image autoencoder frozen.

The evaluation notes matter when comparing releases. Some checkpoints have no benchmark results; others were selected using the same suite reported on their cards. Scores are local measurements unless a card records independent verification.

Model and code licenses are in the individual repositories. Training datasets and external models retain their own terms.