Run an LLM on a VPS

Use a GPU VPS when your laptop lacks VRAM. Start with Ollama for simple installs, or vLLM when you need multi-user throughput.

Quick path (Ollama)

1. Pick a VPS with enough RAM/VRAM for your model (8GB+ for 7B, 16GB+ for larger).
2. Install Docker or Ollama on Ubuntu — see Ollama on Linux or Ollama in Docker.
3. Pull a model: ollama pull llama3.1
4. Restrict API exposure — firewall and SSH tunnel; do not expose publicly without auth.

Production path (vLLM)

For concurrent users and higher tokens/sec on NVIDIA GPUs, install vLLM on Linux. Compare trade-offs in Ollama vs vLLM.

← Get started · Stay on free APIs instead