TensorVizion/SDXL-Lightning-Quantized

SDXL Turbo – Quantized GGUF (Q2, Q4, Q8)

Quantized SDXL Turbo weights in GGUF format for fast, low‑VRAM image generation. This repo provides multiple quantization levels so you can pick the best trade‑off between speed, VRAM usage, and image quality.

Variants * Q2_K – Ultra‑light, minimal VRAM, best for experimentation or very limited GPUs. * Q4_K – Balanced speed/quality, recommended default for most users. * Q8_0 – Near‑FP16 quality, higher VRAM, best for maximum fidelity.

Variant	Format	Approx. Size	Notes
Q2_K	GGUF	~1 GB	Lowest VRAM, fastest, lowest quality
Q4_K	GGUF	~1.8–2 GB	Good balance of quality and speed
Q8_0	GGUF	~2.7 GB	Highest quality, more VRAM needed
Tested on RTX 4060 8 GB and similar GPUs.

---

Usage (Python – llama-cpp-python style backends) pip install --upgrade llama-cpp-python

Example loading (adjust path and variant):

from llama_cpp import Llama

llm = Llama( model_path="sdxl_turbo_q4_k.gguf", n_ctx=4096, n_gpu_layers=-1, # offload as much as possible to GPU )

---

Inference Notes

* For 8 GB GPUs, Q4_K and Q8_0 are both usable; Q2_K is ideal if you want to run other heavy apps in parallel.
* For 4–6 GB GPUs, Q2_K or Q4_K are recommended.
* Higher quantization (Q8_0) preserves more detail and coherence, but uses more VRAM and is slightly slower.

---

License

* Base model: SDXL Turbo under the CreativeML OpenRAIL-M license.
* By using these weights, you agree to the terms of the original SDXL/SDXL Turbo license and any downstream restrictions.

---

Acknowledgements

* Original SDXL Turbo model by Stability AI and contributors.
* Quantization and GGUF conversion by TensorVizion / thomas Barrie.