← Back to articles Tutorial

How to Run a Local LLM (And What It Costs to Run One in the Cloud)

Running a local LLM is one of the fastest-growing trends in AI development. Ollama alone has crossed 13 million monthly searches. Developers want privacy, zero API costs, and the ability to work offline.

But local LLMs aren't free — they're just free in a different way. And cloud LLMs aren't expensive — if you know how to optimize them.

Here's the full picture.

How to run a local LLM

Option 1 — Ollama (easiest)

Ollama is the simplest way to run open-source LLMs locally. Install it, pull a model, and start chatting:

# Install
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama3.2

# Run it
ollama run llama3.2

Ollama runs an OpenAI-compatible API on localhost:11434, which means you can use it with any tool that supports OpenAI — including Tokoscope.

Option 2 — LM Studio (best UI)

LM Studio is a desktop app for running local models with a chat UI. Download from lmstudio.ai, search for a model, download it, and run it. Also exposes an OpenAI-compatible local server.

Option 3 — llama.cpp (most control)

llama.cpp is the underlying engine most local LLM tools use. It runs GGUF-format models directly with full control over quantization, context length, and GPU layers. Higher ceiling, steeper learning curve.

The real cost of local LLMs

Local LLMs feel free because there's no per-token charge. But the real costs are:

• Hardware — a 70B model requires ~40GB VRAM. An RTX 4090 costs $1,500+. A Mac Studio M3 Ultra with enough RAM costs $5,000+.
• Electricity — running a GPU at full load 24/7 costs $50-150/month in electricity depending on your region
• Quality gap — local 7B-13B models are good but not comparable to GPT-4o or Claude Opus for complex tasks
• Maintenance — you manage model updates, compatibility, and performance tuning yourself

Local vs cloud: the actual cost comparison

Scenario	Local LLM	Cloud LLM
10K calls/month, 500 tokens avg	~$30 electricity	~$1.50 (gpt-4o-mini)
100K calls/month, 500 tokens avg	~$50 electricity	~$15 (gpt-4o-mini)
Setup cost	$1,500-5,000 hardware	$0
Privacy	Full — data stays local	Sent to provider
Output quality (complex tasks)	Lower	Higher

For most developers, cloud LLMs are cheaper until you're running millions of calls per month — at which point local infrastructure starts to make economic sense.

The hybrid approach

Many production teams use local LLMs for low-stakes tasks (classification, summarization, simple Q&A) and cloud LLMs for high-stakes tasks (complex reasoning, code generation, customer-facing responses). This gets you the cost savings of local models without sacrificing quality on critical paths.

Tracking usage across local and cloud

Since Ollama and LM Studio both expose OpenAI-compatible APIs, you can wrap them with Tokoscope to track token usage even for local models:

import OpenAI from 'openai'
import { wrap } from 'tokoscope'

// Point to your local Ollama instance
const client = wrap(new OpenAI({
  baseURL: 'http://localhost:11434/v1',
  apiKey: 'ollama' // required but ignored by Ollama
}), {
  apiKey: 'ts_live_...'
})

const res = await client.chat.completions.create({
  model: 'llama3.2',
  messages: [{ role: 'user', content: 'Hello' }]
})

This gives you a unified view of token usage across local and cloud models in the same dashboard.