Private AI

Run large language models locally on your own hardware.

Why Run AI Locally?

Running AI models on your own hardware gives you complete privacy — your data never leaves your machine. No subscriptions, no API calls, no data collection. With modern tools like Ollama and Open WebUI, it's easier than ever to set up.

1 Install Ollama

Ollama is the easiest way to run LLMs locally. One command install:

bash

curl -fsSL https://ollama.com/install.sh | sh

Verify the installation:

bash

ollama --version

2 Pull a Model

Download and run a model. Start with a small one:

bash

ollama pull llama3.2:3b

Or for better performance on capable hardware:

bash

ollama pull llama3.2:7b
ollama pull mistral
ollama pull phi4:14b

3 Run a Model

bash

ollama run llama3.2:3b

You'll get an interactive chat prompt. Type your questions, type /bye to exit.

4 Install Open WebUI

Open WebUI provides a ChatGPT-like interface for your local models. Run it with Docker:

bash

docker run -d -p 3000:8080 \\
  -v open-webui:/app/backend/data \\
  --name open-webui \\
  --restart always \\
  ghcr.io/open-webui/open-webui:main

Access it at http://localhost:3000

5 Useful Ollama Commands

bash

ollama list              # List downloaded models

bash

ollama run llama3.2      # Run a model interactively

bash

ollama rm llama3.2       # Remove a model

bash

ollama cp llama3.2 my-ai # Copy a model

bash

ollama pull llama3.2     # Download a model

Model Recommendations

Model	Size	RAM Req.	Best For
llama3.2:3b	2.0 GB	4 GB	Quick tasks, low-resource
mistral	4.1 GB	8 GB	General purpose, balanced
llama3.2:7b	4.9 GB	8 GB	Better reasoning, coding
phi4:14b	9.1 GB	16 GB	Strong reasoning, math
qwen2.5:7b	4.5 GB	8 GB	Coding, multilingual