Monster-Code/Joker-0.8B-Duplicated

Joker-0.8B 🃏

Hey there! This is Joker-0.8B, a fine-tuned version of Qwen3.5-0.8B designed to be a fast, reliable "System One" decision engine.

I built this because I wanted a small model that could handle classification and reasoning tasks without the overhead of a massive LLM or the hallucination risks of open-ended generation. It’s optimized for speed and calibrated confidence.

What it does

Instead of chatting, Joker is trained to make clear-cut decisions across five key areas:

1. Boolean QA (BoolQ)
2. Topic Classification (AG News)
3. Natural Language Inference (MNLI)
4. Sentiment Analysis (SST-2)
5. Entity Categorization (DBpedia)

How well does it work?

We ran it through the standard benchmarks after 3 epochs of training. Here’s how it stacked up:

Task	Accuracy	Notes
BoolQ	84.8%	Strong reasoning for its size.
AG News	92.5%	Very sharp at topic spotting.
MNLI	84.9%	Holds its own on logical inference.
SST-2	91.9%	Great at picking up sentiment nuances.

It also features a learnable temperature parameter that settled around 0.605, meaning it’s generally quite confident when it makes a call.

How to use it

Standard Text Generation

Since the backbone is just a merged Qwen model, you can use it like any other Hugging Face model:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "Joker-Lab/Joker-0.8B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.bfloat16, 
    trust_remote_code=True,
    device_map="auto"
)

prompt = "The sky is blue. Is this true?"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=10)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Using the Decision Heads

If you want to use the specialized classification heads we trained (for higher accuracy on the tasks above), you’ll need to load the system_one_heads.pt file separately alongside the backbone. This is ideal if you're building a routing agent or a classification pipeline.

Why "Joker"?

Because it’s a wild card in the small-model space—punching above its weight class and keeping things interesting.

Feel free to open an issue if you run into any trouble or want to share what you’ve built with it!