tiny-lm-chat

tiny-lm-chat is an ultra-lightweight, 14-million-parameter Small Language Model based on the Gemma architecture. It has been trained from scratch for conversational efficiency within a compact 192-token context window. It was designed to do singleturn chats in very simple language (input / output).

Evaluation Metrics

• Final Training Loss: 2.601
• Validation Loss (Eval): 2.687
• Perplexity (PPL): ~14.68

Chat Examples (Zero-Shot)

💬 User: What is the meaning of life?
🤖 Bot: To find friends and help others.

💬 User: I cant find my keys.
🤖 Bot: Check your pockets or the door.

💬 User: Are you a human?
🤖 Bot: Nope, just a language.

Recommend generation config

Context Length	192
Temperature	0.75
Top P	0.90
Repeat Penalty	1.10

Usage for chat in terminal

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

def main():
    REPO_ID = "sixf0ur/tiny-lm-chat"
    MAX_CONTEXT = 192

    tokenizer = AutoTokenizer.from_pretrained(REPO_ID)
    model = AutoModelForCausalLM.from_pretrained(REPO_ID, torch_dtype=torch.float32)
    model.eval()

    # custom tokens used for interaction
    BOS_ID  = tokenizer.convert_tokens_to_ids("<bos>")
    USER_ID = tokenizer.convert_tokens_to_ids("<user>")
    BOT_ID  = tokenizer.convert_tokens_to_ids("<bot>")

    print("="*50 + "\n")

    while True:
        user_input = input("User: ").strip()

        if user_input.lower() in ["exit", "quit", "q"]:
            break
        if not user_input:
            continue

        user_token_ids = tokenizer.encode(user_input, add_special_tokens=False)

        prompt_ids = [BOS_ID, USER_ID] + user_token_ids + [BOT_ID]
        prompt_len = len(prompt_ids)

        max_new_tokens = MAX_CONTEXT - prompt_len

        if max_new_tokens <= 5:
            print(f"(Input too long!\n")
            continue

        input_ids = torch.tensor([prompt_ids])

        with torch.no_grad():
            outputs = model.generate(
                input_ids,
                max_new_tokens=max_new_tokens,
                temperature=0.75,
                top_p=0.9,
                repetition_penalty=1.1,
                do_sample=True,
                pad_token_id=tokenizer.pad_token_id,
                eos_token_id=tokenizer.eos_token_id,
            )

        generated_ids = outputs[0][prompt_len:]
        response = tokenizer.decode(generated_ids, skip_special_tokens=True)

        print(f"\nBot: {response.strip()}")
        print("\n" + "="*50)

if __name__ == "__main__":
    main()

lm-studio template (Jinja)

{% for message in messages %}
  {% if message['role'] == 'user' %}
    {{ '<bos><user>' + message['content'].strip() + '<bot>' }}
  {% elif message['role'] == 'assistant' %}
    {{ message['content'].strip() + '<eos>' }}
  {% endif %}
{% endfor %}