ArushBuilds/Pragya-Preview

🧠 Pragya Preview (प्रज्ञा)

Note from the Creator:
"This is my message to all: Pragya Preview is a small edition/prototype of Pragya (Sanskrit for wisdom/intelligence). It was trained on a free Google Colab T4 GPU in a single 1-hour session. I hope you will love it!"

---

Model Summary

Pragya Preview is an experimental, lightweight language model pretrained on the TinyStories dataset. Built as a proof-of-concept under strict compute limits (1-hour single T4 GPU run), it demonstrates rapid baseline convergence and efficient short-form narrative generation.

• Developer: Arush Kumar and Pragya's team
• Model Type: Causal Language Model using Grouped Query Attention(Decoder-only Transformer)
• Language: English (en)
• Base Dataset: roneneldan/TinyStories
• License: Apache 2.0
• Primary Task: Text Generation / Short Story Completion
• Note: Model is not Instruct tuned and is not trained completely

---

Model is open weight

• weights are available in .pt and .onnx

Intended Uses & Scope

Recommended Use

• Fast, low-latency text generation experiments on consumer hardware or CPU environments.
• Educational demonstrations and prototype benchmarking for resource-constrained training.

Limitations

• Capacity: Trained during a brief 1-hour session, so depth and complex reasoning are limited.
• Scope: Tailored primarily for simple, coherent story completion using basic vocabulary.
• Factual Knowledge: Does not possess general world knowledge, advanced coding skills, or multi-turn reasoning abilities.

---

Quickstart & Usage

1. Installation

Ensure you have PyTorch and Hugging Face transformers installed:

pip install torch transformers

2. Inference Code

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Set your Hugging Face model repository ID
model_id = "ArushBuilds/Pragya-Preview"

# Load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id='ArushBuilds/Pragya-Preview',
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

# Define your prompt
prompt = "Once upon a time, "
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

# Generate text
outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    pad_token_id=tokenizer.eos_token_id
)

story = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("\n--- Generated Story ---")
print(story)

---

Training Setup

Parameter	Configuration
Hardware	1x NVIDIA T4 GPU (Google Colab Free Tier)
Training Duration	~1 hour
Dataset	roneneldan/TinyStories
Framework	PyTorch + Hugging Face Transformers

---

Citation & Acknowledgments

If you use or reference Pragya Preview, please credit the underlying dataset:

@article{eldan2023tinystories,
  title={TinyStories: How Small Can Language Models Be and Still Speak Coherent English?},
  author={Eldan, Ronen and Li, Yuanzhi},
  journal={arXiv preprint arXiv:2305.07759},
  year={2023}
}