n8n + Ollama: running an AI agent locally, with no API key

Every call to the OpenAI or Anthropic API costs money, depends on a stable internet connection, and sends the content of your emails, contracts, or support tickets to a third-party server. For a good chunk of the tasks an n8n workflow handles day to day — classification, short summaries, simple extraction — a more modest language model, run locally via Ollama, is more than enough, with no API key and no bill that scales with volume. This guide covers how to connect n8n to Ollama, which use cases it fits, and where the benefits of going local stop.

Why run an LLM locally

Three reasons usually drive this choice, independently of each other:

• Privacy: content never leaves your server. For sensitive data (HR, legal, healthcare), this is sometimes a hard requirement rather than a preference — the topic connects directly to our GDPR audit trail guide, where where processing happens matters as much as how it's logged.
• Predictable cost: no per-request billing. At high volume for simple tasks (email classification, ticket triage), local inference becomes free beyond the cost of the server itself — worth comparing against the figures in our article on tracking AI call costs.
• Network independence: no external calls, so no exposure to the rate limits covered in our 429 errors guide or to a cloud provider's outages.

The trade-off: a reasonably sized local model (7-8 billion parameters, which runs on accessible hardware) still trails well behind GPT-4o or Claude on complex reasoning, output-format reliability, and tool calling. The right move isn't "move everything local" — it's picking the right task for the right building block.

Installing Ollama and picking a model

On Linux, installation is a single command:

curl -fsSL https://ollama.com/install.sh | sh

Native installers exist for macOS and Windows. Ollama starts a local service on port 11434 and exposes an HTTP API compatible with most LangChain clients, including the one n8n uses. From there, pull a model:

ollama pull llama3.1
ollama pull mistral
ollama pull qwen2.5:7b

For classification or summarization tasks — the core of the most common AI workflows on n8n — a quantized 7-8B model is a solid starting point: fast, light, and accurate enough for well-scoped prompts. Save larger models (14B and up) for tasks that genuinely need more nuance, at the cost of higher latency.

Connecting n8n to Ollama

In n8n, the Ollama Chat Model node (AI category, language model sub-nodes) plugs in like any other chat model: onto a Basic LLM Chain for a simple task, or onto an AI Agent for logic involving tools and conversation memory. Its credential only asks for a Base URL, defaulting to http://localhost:11434.

The most common snag: Docker networking. If n8n runs in a container and Ollama runs on the host machine, localhost from inside the container refers to the container itself, not the host. Two common fixes:

• On macOS and Windows (Docker Desktop), use http://host.docker.internal:11434 as the Base URL.
• On Linux, add --add-host=host.docker.internal:host-gateway when launching the n8n container, or run Ollama as a service in the same docker-compose.yml (then referenced by its service name, e.g. http://ollama:11434).

n8n also offers an official Self-hosted AI Starter Kit (n8n + Ollama + Qdrant + PostgreSQL in a single docker-compose) that handles this networking out of the box — a good starting point for testing quickly without configuring everything by hand.

Use cases that fit a local model

Some tasks tolerate a more modest model well and benefit directly from local's free, private nature:

• Simple classification: sorting an email or ticket among a few fixed categories (urgent/normal, client/spam/administrative) — close to the workflow described in our AI email triage guide.
• Short summarization: condensing a discussion thread or a transcript before handing it to a human, with no need for multi-step reasoning.
• Simple field extraction: pulling a date, an amount, or a name from structured text, when the input format is predictable.
• Pre-processing before a stronger model: filtering or rough-sorting a large volume locally (for free), then only sending ambiguous or complex cases to a cloud API — a hybrid setup that caps the bill without sacrificing quality where it counts.

Conversely, an AI Agent with several custom tools or a RAG pipeline with precise citations are, as of today, still more reliable with a model like GPT-4o or Claude: chaining tool calls and staying faithful to sources demands a level of reasoning that reasonably sized local models don't yet deliver consistently.

The real limit: structured output

Several production-oriented n8n workflows rely on a Structured Output Parser to guarantee usable JSON from the LLM (a priority score, a category, extracted fields). That's the case for the email triage and prioritization workflows in the Inbox AI Pack (€79), built on a Basic LLM Chain whose model sub-node is interchangeable. A local 7-8B model generally handles a simple JSON format well, but gets less reliable as the schema grows more complex (nested fields, many enum values): test on a real sample of your data before switching an entire production workflow, and plan for output validation (the Structured Output Parser node fails cleanly when the format doesn't match, which beats a silent error).

A hybrid approach, not a full replacement

Local and cloud aren't mutually exclusive: the same workflow can run an Ollama model for the initial triage of a large batch, then call Claude or GPT via an API key only for the cases flagged as high-priority or ambiguous. This architecture cuts down on billed calls without giving up quality on the decisions that matter — and it fits naturally into a self-hosted instance where you already own the infrastructure.

Going further

Ollama isn't a full AI strategy on its own, but it changes the equation for high-volume, low-complexity tasks: a smaller bill, more privacy, and a good excuse to rethink which steps in your workflows actually need a frontier model. The FlowKit packs are built with interchangeable model sub-nodes: you can start with OpenAI or Anthropic to validate quickly, then progressively migrate the simplest tasks to Ollama once the workflow is stable.