Connect Local LLMs to Claude Code (No Proxy Needed)

Connect Local LLMs to Claude Code

Claude Code talks to Anthropic's API by default, but it doesn't have to. By setting two environment variables, you can point it at a local LLM running via Ollama or LM Studio instead. You won't need a proxy server, or a middleware. What you need is just a direct URL swap.

This gives you:

• Privacy: Your prompts never leave your machine.
• Zero API cost: No Anthropic tokens consumed.
• Offline use: Works without an internet connection.
• Model flexibility: Swap models without changing your workflow.

The only two variables that matter are ANTHROPIC_BASE_URL and ANTHROPIC_API_KEY. Claude Code reads these at startup point them at your local server and it routes all requests there.

How it works

Claude Code is built on top of the Anthropic SDK, which respects the ANTHROPIC_BASE_URL environment variable. Both Ollama and LM Studio expose an OpenAI-compatible and Anthropic compatible REST API at a local port. Since the endpoint shape is compatible, swapping the base URL is all it takes.

Step 1: Install and start your local LLM server

Using Ollama:

brew install ollama
ollama pull qwen2.5-coder:14b
ollama serve

Ollama runs at http://localhost:11434 by default. The /v1 path exposes the OpenAI-compatible API whereas localhost:11434 exposes Anthropic compatible API. See Ollama Anthropic Compatibility for more details.

Using LM Studio:

Download LM Studio from lmstudio.ai, load a model, and start the local server from the Local Server tab. It runs at http://localhost:1234 by default.

Step 2: Set the environment variables

In macOS / zsh:

Open your shell config file:

nano ~/.zshrc

Add these lines at the bottom:

# For Ollama
export ANTHROPIC_BASE_URL=http://localhost:11434/v1
export ANTHROPIC_API_KEY=ollama

Save and reload:

source ~/.zshrc

_Are you using LMStudio?_ The only change is the URL:

# For LM Studio
export ANTHROPIC_BASE_URL=http://localhost:1234/v1
export ANTHROPIC_API_KEY=lmstudio

The `ANTHROPIC_API_KEY` value doesn't matter for local servers as they don't validate it. Set it to anything non-empty so the SDK doesn't complain.

In Windows:

Option A — PowerShell profile (persistent across sessions):

Open your PowerShell profile:

notepad $PROFILE

Add these lines:

# For Ollama
$env:ANTHROPIC_BASE_URL = "http://localhost:11434/v1"
$env:ANTHROPIC_API_KEY = "ollama"

Save and restart your terminal for the changes to take effect.

Option B — System Environment Variables (GUI):

1. Open "Environment Variables"
2. Click "Edit the system environment variables"
3. Click "Environment Variables..."
4. Under User variables, click New and add:
   • Variable: ANTHROPIC_BASE_URL / Value: http://localhost:11434/v1
   • Variable: ANTHROPIC_API_KEY / Value: ollama
5. Click OK and restart your terminal.

Step 3: Run Claude Code

Start Claude Code with --model option

claude --model qwen2.5-coder:14b

It will pick up the environment variables automatically and route all requests to your local LLM server.

Verifying it works

The simplest test is to check the currently selected model by running /model inside the session.

You can now start developing with claude code and should see request activity in ollama or LMStudio logs when Claude Code sends a prompt. If it responds without Anthropic API errors and logs show activity, you're using local model with Claude Code.

Switching models

If you're already inside a Claude Code session, use /model to switch models on the fly without restarting.

To switch back to Anthropic's API, comment the lines from ~/.zshrc and run source ~/.zshrc. On Windows, delete the variables from the Environment Variables panel and restart the terminal. Or unset them temporarily for the current session:

unset ANTHROPIC_BASE_URL
unset ANTHROPIC_API_KEY

A few things to keep in mind

• Local models are significantly weaker than Claude for code reasoning and multi-step tasks. Results will vary depending on the model you pull.
• Claude Code's agentic features (file edits, tool use) still work but the local model needs to support tool-call formatting, which not all models do well.
• If you get `connection refused` errors, make sure your local server is actually running before launching Claude Code.