Local AI (Ollama, LM Studio, and other local backends)

gald3r can run entirely against a model on your own machine — no cloud account, no API key, and (once you've pulled a model) no network connection required. This page is the copy-pasteable setup path for that: point gald3r at a local OpenAI-compatible server (Ollama, LM Studio, vLLM, llama.cpp, or Unsloth Studio) and start running real agent turns against it.

For the full providers.yaml reference (cloud providers, keyring-backed keys, editing verbs, troubleshooting) see providers.md — this page is the local-only fast path and the LAN-endpoint opt-in that page doesn't cover.

Quick setup

# 1. Install and start a local backend. For Ollama:
#    https://ollama.com/download, then pull a model:
ollama pull qwen2.5:7b-instruct

# 2. Generate a starter providers.yaml -- this auto-detects any local backend that's
#    actually reachable right now and writes the models it finds
gald3r init-providers

# 3. Point the default at what you pulled (optional -- --model also works per-call)
gald3r config set default_provider ollama --yes
gald3r config set default_model qwen2.5:7b-instruct --yes

# 4. Confirm gald3r can actually reach it
gald3r providers validate

# 5. Run it
gald3r run "reply with exactly: LOCAL_OK" --model ollama:qwen2.5:7b-instruct

Every command above is a real, shipped gald3r verb — run it as-is. gald3r run/gald3r chat read providers.yaml from the same place init-providers/config/providers write it ($GALD3R_HOME, default ~/.gald3r) — there's no separate directory to keep in sync.

How discovery works

gald3r init-providers probes the default port of every supported local runtime for a live OpenAI-compatible /models endpoint — Ollama (:11434), LM Studio (:1234), vLLM (:8000), and llama.cpp's server (:8080) — and writes a models: entry for whatever it actually finds running, using the real model IDs already pulled/loaded there. Nothing needs to be on your PATH for this to work; it's a plain HTTP probe against each backend's own API, so a service-only deployment (the daemon running with no CLI installed alongside it) is discovered the same way a full local install would be.

Unsloth Studio is the one supported local runtime NOT probed by discovery — its endpoint requires a real API key, so an unauthenticated /models probe can't see it. It gets its own setup path in its section below.

Re-running gald3r init-providers --dry-run any time is itself the verification step — if a backend you just started doesn't show up, confirm it's actually listening on that port (curl http://localhost:11434/v1/models for Ollama, etc.) before checking anything else.

gald3r providers validate gives the same answer for backends already in your providers.yaml, without regenerating the file:

gald3r providers validate

[OK  ] ollama
         ok: API key resolved via local
         ok: reachable (http://localhost:11434/v1/models)

See providers.md for a per-backend recipe (install, pull/load a model, verify) covering all four supported local runtimes.

Adding a model, or a whole new provider

gald3r providers model add ollama gemma3:27b --role vision --context-window 32768

providers model add/providers role set/providers list (all documented in providers.md) work exactly the same for local providers as cloud ones — there's no local-specific subset of the edit surface.

Running against a local model

--model <provider>[:<model_or_alias>] selects any configured provider on a per-call basis, overriding default_provider/default_model:

gald3r run "list the files in this directory" --model ollama:qwen2.5:7b-instruct
gald3r chat --model lmstudio:qwen2.5-coder-14b

A local model goes through the exact same session engine, tool-calling loop, and stream-json protocol events as a cloud model — nothing about gald3r run/gald3r chat changes based on where the model is served from.

Unsloth Studio (keyed local backend)

Unsloth Studio is supported as a fifth local runtime with a dedicated unsloth provider id — but unlike the four runtimes above, its OpenAI-compatible endpoint is not keyless, which is why gald3r init-providers doesn't auto-discover it. Setup is three steps:

# 1. Start Studio's API server (every concrete example in Unsloth's own API guide
#    uses port 8888, so that's the default gald3r ships)
unsloth studio -p 8888

# 2. Generate an API key in Studio's Settings -> API (format: sk-unsloth-...), then
#    export it under the exact variable name both Unsloth's docs and gald3r use:
#      PowerShell:  $env:UNSLOTH_API_KEY = "sk-unsloth-..."
#      bash/zsh:    export UNSLOTH_API_KEY="sk-unsloth-..."

# 3. The starter providers.yaml written by `gald3r init-providers` already ships an
#    `unsloth:` block (base_url http://localhost:8888/v1, api_key $UNSLOTH_API_KEY) --
#    verify reachability, then run against whatever model Studio has loaded:
gald3r providers validate
gald3r run "reply with exactly: LOCAL_OK" --model unsloth:<model-id-from-studio>

A missing key fails at call time with a real auth error — there is no keyless placeholder for unsloth the way ollama/lmstudio have one. If your Studio instance booted on a different port, set UNSLOTH_BASE_URL (or edit the block's base_url). The full reference — the providers.yaml block shape, the port-sourcing rationale, and how Unsloth's own unsloth start <agent> wrapper relates to gald3r's provider registry — is in providers.md.

LAN endpoints and .gald3rsecret's no-cloud tier

A project can drop a .gald3rsecret file at its root to mark some paths no-cloud: — readable by a local model, but never allowed to reach a remote/cloud provider, a persisted transcript, or any other outbound log. Whenever such a file is in play, gald3r needs to know whether the model you're currently running against actually counts as "local":

• A backend reached over a loopback address (localhost, 127.0.0.1, ::1) is automatically treated as local — no configuration needed. This covers the default setup above (Ollama/LM Studio/vLLM/llama.cpp all bind to loopback by default).
• A backend on another machine on your network — a second box running Ollama, reached over its LAN IP or hostname — is not local by default, even though it's still a model you control. no-cloud:-tiered files stay refused until you explicitly opt that host in.

To opt a LAN endpoint in, add a local-endpoints: line to your project's .gald3rsecret:

no-cloud: secret_local.txt
local-endpoints: 192.168.1.50

A bare host (no port) matches that host on any port; local-endpoints: mybox.local:11434 restricts the allowlist entry to that exact port instead. One host per line, same as .gald3rsecret's other tier lines. Without an entry here, gald3r fails closed — a no-cloud: file stays unreadable to a non-loopback backend rather than silently trusting it.

Where next

• providers.md — the full providers.yaml reference: cloud providers, keyring-backed keys, every gald3r providers edit verb, and provider-error troubleshooting
• concepts.md — the one-paragraph summary of how gald3r talks to models at all
• verbs.md — every provider-related verb in the full command map
• troubleshooting.md — gald3r doctor's providers line and the general troubleshooting flow