Instructions to use layer8labs8/OrdoPy-Coder-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use layer8labs8/OrdoPy-Coder-0.5B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="layer8labs8/OrdoPy-Coder-0.5B") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B") model = AutoModelForCausalLM.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use layer8labs8/OrdoPy-Coder-0.5B with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "layer8labs8/OrdoPy-Coder-0.5B" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B
- SGLang
How to use layer8labs8/OrdoPy-Coder-0.5B with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "layer8labs8/OrdoPy-Coder-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "layer8labs8/OrdoPy-Coder-0.5B" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "layer8labs8/OrdoPy-Coder-0.5B", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use layer8labs8/OrdoPy-Coder-0.5B with Docker Model Runner:
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B
OrdoPy-Coder-0.5B (v2)
OrdoPy-Coder-0.5B v2 is Qwen/Qwen2.5-Coder-0.5B-Instruct fine-tuned with a rejection-sampling recipe: the base model wrote 4 solutions for each of 2,000 KodCode problems, every solution was executed against the problem's unit tests, and only passing solutions were kept (max 2 per problem; GPT-4o reference solutions filled in for unsolved problems, capped at 20% of the data). A LoRA adapter (r=32, all linear layers) was trained on these 1,106 examples and merged; the epoch was selected on MBPP train + validation, never on test data. It runs in about 1 GB of VRAM (float16).
Benchmarks (EvalPlus, greedy pass@1)
Base and v2 were scored by the official EvalPlus harness in the same run (Kaggle 2x T4, float16):
| Benchmark | Base, officialยน | Base, same run | v2 (this model) | v1 (previous upload) |
|---|---|---|---|---|
| HumanEval | 61.6 |
57.9 |
61.0 |
48.8 |
| HumanEval+ | 57.3 |
53.7 |
55.5 |
46.3 |
| MBPP | 52.4 |
52.1 |
51.1 |
48.4 |
| MBPP+ | 43.7 |
44.2 |
41.0 |
42.1 |
| Average | โ | 52.0 |
52.2 |
46.4 |
ยน Qwen2.5-Coder Technical Report, Table 16 (Qwen's own evaluation setup).
Honest summary: v2 is clearly better than v1 (+5.8 points on average) and about on par with the base model (better on HumanEval / HumanEval+, worse on MBPP / MBPP+; single seed, differences of this size are close to run-to-run noise). It does not beat the base model overall.
About v1. The first upload is kept under the
v1tag of this repository (AutoModelForCausalLM.from_pretrained("layer8labs8/OrdoPy-Coder-0.5B", revision="v1")). It was a supervised fine-tune that scores below the base model on all four benchmarks. Its card claimed HumanEval9.76% -> 40.85%; that comparison was a measurement error (a 224-token generation cap truncated the base model's answers, and the checkpoint was selected on test slices).eval_benchmarks.jsonin this repository is v1 training telemetry. Full v2 report and code: OrdoTrain (results/ordopy_rft_lora_report.json).
Model Details
- Architecture:
Qwen2ForCausalLM(24 layers, hidden size 896, 494M parameters, 32K context) - Base model:
Qwen/Qwen2.5-Coder-0.5B-Instruct - Fine-tuning: LoRA r=32 / alpha=64 on all linear layers, lr 1e-4, 1 epoch kept, assistant-only loss, merged into the weights
- Format:
model.safetensors(float16) + Qwen ChatML tokenizer
Quickstart
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "layer8labs8/OrdoPy-Coder-0.5B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.float16, device_map="auto")
messages = [{"role": "user", "content": "Write a Python function `is_palindrome_clean(s: str) -> bool` that ignores spaces and punctuation."}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
On transformers < 4.56 use torch_dtype= instead of dtype=.
- Downloads last month
- 140
Model tree for layer8labs8/OrdoPy-Coder-0.5B
Base model
Qwen/Qwen2.5-0.5B
docker model run hf.co/layer8labs8/OrdoPy-Coder-0.5B