Instructions to use guinmoon/LLMFarm_Models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use guinmoon/LLMFarm_Models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf guinmoon/LLMFarm_Models:Q4_K_M # Run inference directly in the terminal: llama cli -hf guinmoon/LLMFarm_Models:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf guinmoon/LLMFarm_Models:Q4_K_M # Run inference directly in the terminal: llama cli -hf guinmoon/LLMFarm_Models:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf guinmoon/LLMFarm_Models:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf guinmoon/LLMFarm_Models:Q4_K_M
Use Docker
docker model run hf.co/guinmoon/LLMFarm_Models:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use guinmoon/LLMFarm_Models with Ollama:
ollama run hf.co/guinmoon/LLMFarm_Models:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use guinmoon/LLMFarm_Models with Docker Model Runner:
docker model run hf.co/guinmoon/LLMFarm_Models:Q4_K_M
- Lemonade
How to use guinmoon/LLMFarm_Models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull guinmoon/LLMFarm_Models:Q4_K_M
Run and chat with the model
lemonade run user.LLMFarm_Models-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Commit History
-
Upload gemma 2b_it_v2_Q6_K.gguf
Verified signature14bf374 -
Upload gemma 2b_it_v2_IQ3_XS.gguf
Verified signature1593db0 -
Upload 2 files
Verified signatured36a75c
-
Upload gemma-2b-it-Q5_K_M.gguf
Verified signature9910a5c -
Upload gemma-2b-it-IQ3_S.gguf
Verified signature36bb3df -
Upload gemma-2b-it-IQ4_NL.gguf
Verified signature3e68764 -
Upload gemma-2b-it.Q8_0.gguf
Verified signaturedb3dd83
-
Upload open_llama_3b_v2_Q8_0.gguf
a244ba3 -
Upload stablelm-3b-4e1t-Q4_K_M.gguf
d853a98 -
Upload open_llama_3b_v2_Q4_K.gguf
7e80aa6 -
Delete stablelm-tuned-alpha-3b-ggml_v3-q5_1.bin
2017b17
-
Delete openllama-3b-v2-q4_0.gguf
b3afe50
-
Upload q4_1-RWKV-4-Raven-1B5-v12-Eng98%-Other2%-20230520-ctx4096.bin
c098339
-
Upload 2 files
a4c7ba8 -
Delete dolly-v2-3b-ggml_v3-q5_1.bin
8df4276 -
Delete rp-incite-base-v1-3b-ggmlv3-q5_1.bin
d225c44 -
Delete orca-mini-3b.ggmlv3.q4_1.bin
e29ff33 -
Delete orca-mini-3b.ggmlv3.q5_1.bin
f24b9fb -
Delete open-llama-3b-1T_q5_1.bin
a92e31b
-
Delete AI-Dungeon-2-Classic.bin
crash on iosc5083f2 -
Delete orca-mini-v2_7b-q3_K_S.gguf
crash on ios4648b90
-
Upload orca-mini-v2_7b-q3_K_S.gguf
9d1db97 -
Upload 2 files
llama 2 7b gguf orca 3b ggufb371dfc
-
Upload 3 files
58bcf25
-
Upload orca-mini-3b.ggmlv3.q5_1.bin
f1d74d5
-
Upload replit-code-v1-3b-ggml-q5_1.bin
24d2ad8 -
Upload open-llama-3b-1T_q5_1.bin
9280ca8 -
Delete openllama-3b-350bt-ggml_v3-q4_0.bin
4c6ec20
-
Upload stablelm-tuned-alpha-3b-ggml_v3-q5_1.bin
5eede0c -
Upload rp-incite-base-v1-3b-ggmlv3-q5_1.bin
68688e7 -
Delete stablelm-tuned-alpha-3b-ggml_v3-q5_1.bin
4e9e724
-
Upload pythia-70m-ggml_v3-q5_1.bin
847560d -
Upload stablelm-tuned-alpha-3b-ggml_v3-q5_1.bin
967efad -
Upload pythia-2.8b-ggml_v3-q5_1.bin
4a0c2a8 -
Upload openllama-3b-350bt-ggml_v3-q4_0.bin
9eb127a -
Upload dolly-v2-3b-ggml_v3-q5_1.bin
7efaf4f -
initial commit
3e6156e