Run a Local LLM Tonight: Ollama in 20 Minutes

Your cloud AI subscription does three things a local model doesn’t: it sends your prompts to someone else’s servers, it meters every token, and it stops working when the internet does. Ollama flips all three. It’s a free, open-source runtime that downloads open-weight models and runs them entirely on your machine — a Mac, a Windows PC, or a Linux box. The install takes about two minutes. The model download is the long pole, and picking the right model for your hardware is the actual skill. Total time tonight: about twenty minutes.

How this guide was made: the install commands and model pairings below come from Ollama’s own documentation and current third-party guides checked October 21, 2026. Models and sizes change — the library page for each model shows its current download size before you pull.

Why run a model locally

Three reasons, in order of how often they matter:

1. Privacy. Drafts, code, and documents never leave your machine. For anything you’d hesitate to paste into a cloud chatbot — client work, unreleased ideas, personal notes — the local model is the only one you fully control.
2. Zero marginal cost. No subscription meter, no per-token billing. Run a thousand prompts a day or one; the cost is the same hardware you already own.
3. Offline. Planes, cabins, dead zones, outages — the model is a file on your disk. It works wherever your computer works.

None of this replaces the frontier models. Claude, ChatGPT, and Gemini remain stronger on hard reasoning and up-to-date knowledge. Think of it as an autonomy spectrum, not a replacement ladder: a local model is a capable assistant that lives on your machine, runs agents and tool calls locally, and keeps everything private. Cloud models are the heavier brains you reach for when the task exceeds your hardware. Both directions are useful; tonight we’re adding the private end of the spectrum.

Step 1: Install Ollama

macOS — download the app from ollama.com/download, or install via Homebrew:

brew install ollama

Linux — run the official install script:

curl -fsSL https://ollama.com/install.sh | sh

Windows — download the installer from ollama.com/download, or in PowerShell:

irm https://ollama.com/install.ps1 | iex

Ollama runs as a background service after installation. Verify it with ollama --version in a terminal. No account is required — Ollama is free and open source.

Step 2: Pick a model that fits your machine

This is the step that matters. The rule of thumb: plan on roughly 1 GB of memory per billion parameters, and check the model’s library page for its actual download size before pulling. The download size should sit comfortably below your available VRAM (on a GPU PC) or unified memory (on a Mac), with room left over for the context window.

Current pairings, from Ollama’s library as of October 2026:

Model tag	Maker	Download size	Fits
gemma4:e2b	Google	about 5–7 GB	8 GB VRAM, or a 16 GB Mac — Google’s smallest Gemma 4, with multimodal, tool calling, and reasoning built in
gemma4 (E4B default)	Google	about 7–10 GB	16 GB VRAM, or a 24–32 GB Mac — the well-rounded generalist
gpt-oss:20b	OpenAI	about 14 GB	16 GB VRAM / 24 GB+ Mac — OpenAI’s open-weight model, strong on reasoning and agentic tasks
qwen3.8:27b	Alibaba Qwen	about 18 GB	24 GB VRAM or larger Macs — the current best-overall on consumer hardware
muse-glimmer:30b	Meta	about 18 GB	24 GB VRAM or larger Macs — multimodal, long context
gpt-oss:120b	OpenAI	about 65 GB	Workstations with 80 GB+ — frontier-class, not a laptop model
qwen3-coder:30b	Alibaba Qwen	about 19 GB	24 GB+ VRAM — the coding pick; for lighter machines, qwen3:8b handles small edits
deepseek-r1:7b	DeepSeek	about 5 GB	Light machines — a reasoning model distilled small enough to run almost anywhere
llama3.1:8b	Meta	about 5 GB	Most laptops — one of the most-pulled models in Ollama history; the safe default everyone’s tutorial uses

One tag warning: some library entries now carry a “cloud” tag (like gpt-oss:120b-cloud). Those run on Ollama’s servers, not your machine — the same API shape, but not local. Check the tags on the model’s library page before you pull if “runs on my hardware” is the point.

If you’re shopping for hardware specifically to run larger models, that’s exactly what the two hardware guides compare: prebuilt machines for local LLMs and custom PC builds for local LLMs.

Step 3: Pull and run it

One command does everything — downloads the model if you don’t have it, loads it, and opens a chat session:

ollama run gemma4:e2b

(The first run takes as long as the download: about 5–7 GB on a decent connection is the bulk of your twenty minutes.) When you see the >>> prompt, you’re talking to a model running on your own hardware. Type /bye to exit.

Three more commands cover the rest of model management:

ollama pull qwen3.8:27b   # download without starting a chat
ollama list               # show what's on your machine
ollama rm <model>         # delete a model to free disk space

Models live on your disk (typically ~/.ollama), so plan 10–50 GB of free space depending on how many you keep around.

Step 4: Make it useful

The terminal chat is the proof it works. Two upgrades turn it into a daily tool:

A web UI. Open WebUI is the popular open-source ChatGPT-style interface that talks to Ollama. Same models, a proper chat UI, conversation history — no terminal required for everyday use.

The API. Ollama exposes an OpenAI-compatible API at http://localhost:11434. Point coding assistants and agent tools at it (VS Code extensions like Continue, or Ollama’s own coding-agent integrations) and your local model becomes the engine behind a private coding agent. Nothing in the loop leaves your machine.

What local models won’t do for you

Honest boundaries, so tonight’s experiment doesn’t oversell itself:

• Frontier reasoning stays in the cloud. On genuinely hard problems — novel math, deep research synthesis, multi-step planning — the hosted models still win. Local models are the right tool for drafting, summarizing, coding assistance, and anything private.
• Knowledge has a cutoff. A downloaded model doesn’t update itself. It knows what it knew at training time, and it won’t browse the web on its own.
• The first model you try is a starting point. The library is deep — coding models, vision models, tiny edge models. Try two or three over a week; the switching cost is one command.

None of those are reasons to skip tonight’s install. They’re reasons the local model earns a permanent slot next to the cloud subscription rather than replacing it.

Frequently asked questions

Is Ollama really free? Yes. The software is free and open source, no account required for local use. The models in the library are open-weight and free to download. Your only costs are the hardware you already own and the electricity it draws.

How much disk space do I need? The install itself is under 200 MB. Models range from about 1 GB (tiny) to 65 GB (the big ones). A realistic starter setup — one or two general models — fits in 15–25 GB.

Will it run on my 8 GB laptop? Yes, with the small end of the library: llama3.2:1b, gemma3:270m, or deepseek-r1:1.5b are designed for constrained machines. They won’t impress you the way a frontier model does, but they run, they’re private, and they’re the cheapest way to learn the tooling.

Mac, Windows, or Linux — which is best for Ollama? All three are officially supported. Apple Silicon Macs punch above their weight because unified memory lets the GPU use the full RAM pool — a 24 GB Mac runs models a 16 GB VRAM PC can’t hold. Linux with an NVIDIA GPU gives the most control; Windows works fine through the native installer.

Can I use it like an API in my own scripts? Yes — http://localhost:11434 serves a REST API and an OpenAI-compatible endpoint while Ollama runs. That’s the doorway to agents, automations, and app integrations running on local models.

What about the cloud models in the library? Tags ending in -cloud run on Ollama’s hosted service after a sign-in — same API shape, not local. They’re convenient for oversized models, but they reintroduce the server round-trip that local use avoids. For this guide’s purpose, stick to local tags.