Ollama lets you run open-source LLMs with a single command. The hard part isn't running them — it's knowing which model to pull for your use case.
Here's the definitive breakdown for 2026.
# Install Ollama curl -fsSL https://ollama.com/install.sh | sh # Pull and run any model ollama pull llama3.2 ollama run llama3.2 # List what you have ollama list
Meta's Llama 3.2 is the best all-around local model for most users. The 3B version runs on any modern laptop (4GB RAM). The 8B version needs 8GB RAM but is noticeably smarter. Both are fast and follow instructions well.
ollama pull llama3.2 # 3B — fastest, least RAM ollama pull llama3.2:8b # 8B — better quality
Alibaba's coding-specialized model outperforms Llama 3 on code tasks. The 7B version is a great balance of speed and quality. The 32B version is near-frontier for local models but needs 20GB VRAM.
ollama pull qwen2.5-coder # 7B ollama pull qwen2.5-coder:32b # 32B — best quality
DeepSeek's R1 model is specifically trained for step-by-step reasoning. Excellent for math, logic puzzles, and complex analysis. The distilled 7B version runs locally and outperforms much larger general models on reasoning tasks.
ollama pull deepseek-r1:7b ollama pull deepseek-r1:14b # if you have the VRAM
Google's Gemma 2 handles long documents well and has strong instruction following. Good choice for summarization tasks on local hardware.
ollama pull gemma2 ollama pull gemma2:27b # larger context, better quality
If you're on a machine with under 8GB RAM, Mistral 7B remains the best option. Fast, reliable, and capable enough for most everyday tasks.
ollama pull mistral
| RAM available | Max model size | Recommended model |
|---|---|---|
| 4GB | 3B | llama3.2:3b, phi3:mini |
| 8GB | 7-8B | llama3.2, mistral, qwen2.5-coder |
| 16GB | 13B | llama3.2:13b, deepseek-r1:14b |
| 32GB | 30B | qwen2.5-coder:32b, gemma2:27b |
| 64GB+ | 70B | llama3.3:70b, qwen2.5:72b |
Ollama exposes an OpenAI-compatible API at localhost:11434, so you can use it as a drop-in replacement in any OpenAI-based code:
import { wrap } from 'tokoscope'
import OpenAI from 'openai'
const client = wrap(new OpenAI({
baseURL: 'http://localhost:11434/v1',
apiKey: 'ollama' // required but ignored
}), { apiKey: 'ts_live_...' })
const res = await client.chat.completions.create({
model: 'llama3.2',
messages: [{ role: 'user', content: 'Hello' }]
})
Tokoscope works with any OpenAI-compatible endpoint. Free to start.
Get started free →