llama.cpp vs Ollama
Both are catalogued under Model Serving. The figures come from the GitHub API; the assessments are ours.
llama.cppInference in portable C++, from laptops to single-board machinesOfficialMIT
OllamaRun open models locally with one commandOfficialMITAt a glance
| At a glance | llama.cpp | Ollama |
|---|---|---|
| License | MIT | MIT |
| Languages | C++, C | Go |
| Deployment | Runs locally / Self-hosted | Runs locally / Self-hosted |
| Maturity | Established | Established |
| Stars | 124k | 179k |
| Star growth over the last 7 days | +30 ★ | +21 ★ |
| Forks | 21.7k | 17.4k |
| Open issues | 2k | 3.7k |
| Last commit | 15 Aug 2026 | 15 Aug 2026 |
| Activity | Active | Active |
What each one does
llama.cpp
The quantisation work here is what put usable models on hardware without a datacentre GPU, and much of the local-model ecosystem is built on top of it. Working with it directly means dealing with build flags and hardware specifics that higher-level wrappers hide from you.
Full entry →Ollama
Handles model download, quantisation and an OpenAI-compatible server so a local model is a single command away. The obvious starting point for development against local inference. For serving many concurrent users, a throughput-oriented server will do considerably better.
Full entry →What you can do
llama.cpp
- Run a model in one command — llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF pulls the GGUF straight from Hugging Face and drops you into an interactive session, with no Python environment involved.
- Serve an OpenAI-compatible endpoint — llama serve -hf <repo> starts a REST server that existing OpenAI clients can be pointed at, and ships a built-in web UI for trying the model.
- Fit models larger than VRAM — Integer quantisation from 1.5-bit through 8-bit, combined with CPU+GPU hybrid inference, lets you offload only part of a model to the GPU and keep the rest in system RAM.
- Target the hardware you have — One source tree covers Metal and ARM NEON on Apple silicon, CUDA on NVIDIA, HIP on AMD, MUSA on Moore Threads, plus Vulkan, SYCL, OpenCL, WebGPU and CANN backends.
- Constrain output with GBNF grammars — A GBNF grammar file forces generation to match a syntax you define, which is how you get reliably structured output out of a small local model.
Ollama
- Run a model in one command —
ollama run gemma4pulls the weights and opens a chat session, with the full catalogue of runnable models at ollama.com/library. - Back a coding agent locally —
ollama launch claudestarts the Claude Code integration against local models, and the same command covers Codex, Copilot CLI, OpenCode, Droid and DeepSeek Harness. - Call it from your code — A REST endpoint at
http://localhost:11434/api/chataccepts JSON withmodel,messagesandstream, and official clients install withpip install ollamaornpm i ollama. - Turn it into an assistant —
ollama launch openclawconnects the local model to WhatsApp, Telegram, Slack and Discord as a personal AI assistant.