Install vLLM on Linux (NVIDIA GPU)

Last verified: 2026-09-06

1. Use Ubuntu 22.04+ with an NVIDIA GPU, recent drivers, and CUDA 12.x
2. Create a Python 3.10+ venv: python3 -m venv .venv && source .venv/bin/activate
3. Install vLLM: pip install vllm
4. Serve an OpenAI-compatible API: vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8000
5. Test: curl http://localhost:8000/v1/models
6. Point clients at http://localhost:8000/v1 with any placeholder API key
7. For multi-GPU, add --tensor-parallel-size N matching your GPU count
8. Prefer a GPU VPS when local VRAM is not enough — see /en/guides/run-llm-on-vps/ and /en/compare/ollama-vs-vllm/

Back to catalog