Running an LLM on a VPS without a GPU using Ollama: a realistic guide

Running a language model on your own server has an obvious benefit: the data you send to it never leaves the machine, and the cost no longer depends on the number of requests. But most guides assume you have a graphics card. On a standard VPS without a GPU, what can you really expect?

The short answer: much more than you might think for some uses, much less than you might hope for others. This guide gives concrete orders of magnitude so you can decide before ordering, then shows how to install Ollama and use it from n8n.

1. Why an LLM can run without a GPU

Two techniques make CPU execution possible. The first is quantization: model parameters, originally stored on 16 bits, are compressed to about 4 bits. The model then takes nearly four times less memory, with a moderate quality loss on most tasks. The second is the llama.cpp engine behind Ollama, optimized for the vector instructions of modern processors.

On a CPU, the limiting factor is less raw computing power than how fast memory transfers data to the processor. To generate each word, the engine has to go through all the model's parameters. The bigger the model, the longer this takes, and beyond a certain number of cores, adding more changes almost nothing.

2. How much RAM you need

The model must fit entirely in RAM, with room left for the system, the conversation context and other services. With the most common 4-bit quantization, the needs are roughly:

If n8n and PostgreSQL run on the same server, add 2 to 4 GB. A long context, such as a whole document passed to the model, also uses memory on top of the parameters.

3. What speed to expect

Speed is measured in tokens per second, a token being about three quarters of a word on average. Published CPU measurements give these orders of magnitude:

• 3-billion-parameter model: around 15 to 30 tokens per second on 6 to 12 vCPUs. The answer appears at a comfortable reading speed.
• 7 to 8 billion: around 5 to 15 tokens per second on 8 to 12 vCPUs. A 300-token answer takes between 20 seconds and a minute.
• 14 billion: only a few tokens per second. Usable for overnight processing, not for interaction.

A second delay often surprises people: before producing the first word, the model has to read all the text provided. On a CPU, a document of several pages can take tens of seconds before the answer starts. For long texts, split them or summarize in stages.

Finally, a CPU server efficiently handles one request at a time. Ten simultaneous users will not get ten fast answers, but a queue.

4. What works and what does not

Good uses

• Classifying and sorting: assigning a category to an email, a support ticket or a customer review. The answer is only a few tokens, so it arrives quickly.
• Extracting data: turning an email or an invoice into structured JSON.
• Background summaries: summarizing documents or reports overnight, when nobody is waiting.
• Generating embeddings: these models are small and fast on CPU, and they are the basis of internal document search.
• Handling sensitive data: pseudonymizing, analyzing or rewriting content that must not go to an external provider.

Uses to avoid

• Real-time chat for several users: the queue quickly becomes frustrating.
• Complex reasoning or advanced code generation: small models are clearly weaker here than large models available through an API.
• Analyzing long documents on the fly: the initial reading phase becomes too long.

5. Install Ollama on the VPS

With Docker, installation takes one command. Note the 127.0.0.1 address before the port: Ollama has no authentication and must never be exposed directly to the Internet.

docker run -d --name ollama \
  --restart unless-stopped \
  -p 127.0.0.1:11434:11434 \
  -v ollama:/root/.ollama \
  ollama/ollama

Then download a model and check that it responds:

docker exec -it ollama ollama pull llama3.2:3b
curl http://127.0.0.1:11434/api/generate -d '{
  "model": "llama3.2:3b",
  "prompt": "Classify this message as urgent or not urgent: the server is not responding.",
  "stream": false
}'

By default, Ollama unloads the model after a few minutes of inactivity, and reloading adds several seconds to the next request. If the server is dedicated to this use, keep it loaded by adding -e OLLAMA_KEEP_ALIVE=24h to the run command.

6. Connect Ollama to n8n

If you run n8n with Docker Compose, the simplest option is to add Ollama as a service in the same file. Both containers then share a private network, with no open port:

  ollama:
    image: ollama/ollama
    restart: unless-stopped
    environment:
      OLLAMA_KEEP_ALIVE: 24h
    volumes:
      - ollama_data:/root/.ollama

Add ollama_data: to the volumes section at the bottom of the file, restart with docker compose up -d, then download the model with docker compose exec ollama ollama pull llama3.2:3b.

In n8n, create an Ollama credential with the address http://ollama:11434. You can then use the Ollama Chat Model node as the model of an AI Agent node or an LLM chain.

An effective pattern combines both: the local model sorts, extracts and handles anything sensitive or repetitive, and only complex requests go to a model reached through an API.

7. API or local model: how to decide

For raw answer quality, large models available through an API remain far ahead of what a CPU can run, and at moderate volumes their cost per request is often low. A local model is justified for three specific reasons:

• confidentiality, when data must not leave your infrastructure;
• volume, when thousands of small repetitive tasks would make the API bill significant;
• independence, when you do not want to depend on a provider's availability or price changes.

If none of these applies, start with a model through an API: any VPS can run the workflows that call it. If one does, a 16 GB server with a 7 to 8 billion parameter model is the most balanced starting point. See our NVMe VPS plans.

Frequently asked questions