local-llm-claude-code

Name: local-llm-claude-code
Author: mjunaidca

26stars11forks

How to install local-llm-claude-code?

mjunaidca/mjs-agent-skills/local-llm-claude-code

$npx -y skills add mjunaidca/mjs-agent-skills --skill local-llm-claude-code

Installs into the current project.

Summary

Set up a FREE local LLM (via Ollama) and connect Claude Code to it so it runs offline with no API costs. Use this whenever the user wants to run Claude Code on a local/self-hosted model, use Ollama/Qwen/Llama/Mistral/DeepSeek with Claude Code, get a "free" or "offline" coding age

Files · 1

View on GitHub

SKILL.md

1	# Local LLM → Claude Code
2
3	Connect Claude Code to a locally-served open model. Since **Ollama v0.14+ natively speaks the
4	Anthropic Messages API (`/v1/messages`), this needs no proxy** — just three env vars.
5
6	## The one thing to say up front (set expectations honestly)
7
8	Making this work and making it usable are different. Two independent thresholds must BOTH
9	be cleared, and cheap hardware usually misses both:
10
11	1. Capability — the model must emit valid structured tool calls, turn after turn.
12	Small models (≤1.7B) malform them (`args` object vs string, skipped tools). Fixed by a
13	bigger model (≥14B, ideally a 30B-class MoE), NOT by faster hardware.
14	2. Throughput — the box must chew Claude Code's ~18,000-token system prompt every turn
15	fast enough to beat its request timeout. CPU-only = minutes/turn. Fixed by a GPU, NOT
16	by a smarter model.
17
18	Say this to the user before installing, then read `references/hardware-sizing.md` and run
19	`scripts/check-hardware.sh` to give them a specific, honest verdict for their box.
20
21	## Workflow
22
23	Run these in order. Each script is idempotent and needs no root (user-space install).
24
25	### 1. Reality-check the hardware
26	```bash
27	bash scripts/check-hardware.sh
28	```
29	Reports CPU/RAM/GPU/disk and recommends a model tier (or warns it'll only be a learning rig).
30	Details + the sizing table: `references/hardware-sizing.md`.
31
32	### 2. Install Ollama (lean, user-space)
33	```bash
34	bash scripts/install-ollama.sh
35	```
36	Streams the release tarball through `zstd` and excludes the CUDA/ROCm/MLX GPU libraries,
37	so a CPU box installs in ~112 MB instead of ~3 GB. Starts `ollama serve` on `:11434`.
38	(On a GPU box, drop the `--exclude` flags — see the script's header.)
39
40	### 3. Pull + tune a model
41	```bash
42	bash scripts/make-model.sh <base-model> <derived-name>
43	# e.g. bash scripts/make-model.sh qwen3:14b qwen3-cc
44	```
45	Pulls the base model and builds a derived model with a Modelfile that sets
46	`num_ctx` large enough for Claude Code's ~18K prompt (default 32768). This is the #1 cause
47	of "local Claude Code is broken" — the Ollama default of 4096 silently truncates the prompt.
48
49	### 4. Launch Claude Code against it
50	```bash
51	bash scripts/launch.sh <derived-name> # e.g. qwen3-cc
52	```
53	Exports the three env vars and runs `claude --model <name>`. The wiring:
54	```bash
55	export ANTHROPIC_BASE_URL="http://localhost:11434"
56	export ANTHROPIC_AUTH_TOKEN="ollama" # required but ignored locally
57	export ANTHROPIC_API_KEY="" # MUST be empty so the token is used
58	```
59	On slow (CPU) boxes the launcher also sets `API_TIMEOUT_MS=900000` so turns don't 500 mid-prefill.
60
61	### 5. Verify end-to-end
62	```bash
63	bash scripts/verify.sh <derived-name>
64	```
65	Tests, in order: native `/v1/messages` chat → a real tool call → `count_tokens` (a 404 here
66	is FINE on v0.14+, not a hang) → measures prompt-eval tok/s so you can predict per-turn latency.
67
68	## Hard-won gotchas (all verified the hard way)
69
70	- No proxy needed. Ollama v0.14+ serves the Anthropic API natively. Do NOT install LiteLLM
71	or claude-code-router for Ollama. (vLLM/LM Studio DO need a proxy — see `references/other-backends.md`.)
72	- `num_ctx` ≥ 32768. Claude Code's system prompt is ~18K tokens. The 4096 default breaks it.
73	- Reasoning models (Qwen3, DeepSeek-R1) think by default and it's slow. You CANNOT disable
74	thinking through the Anthropic `/v1/messages` path (no `think` field passes; there's no
75	`PARAMETER think` in Modelfiles; template edits don't change server-side parse). For Claude
76	Code, either accept the overhead or choose a non-reasoning model. For YOUR OWN agents, call
77	Ollama's native `/api/chat` with `"think": false` — that DOES disable it (big speed win).
78	- `count_tokens` returns 404 on v0.14+ — harmless; Claude Code falls back to local estimation.
79	A hang (not a 404) means your Ollama is too old — upgrade.
80	- Prompt cache helps a lot. Turn 2+ reuses the cached ~18K system-prompt prefix, so only the
81	first turn pays full prefill. Any context growth past the cached prefix re-triggers eval.
82	- Install lean or run out of disk. The full tarball unpacks to ~3 GB (GPU libs); stripping
83	them → ~112 MB. Critical on small VPSs.
84
85	## When the us

People who liked this also viewed

6 picks

Type

local-llm-claude-code

26stars11forks

View on GitHub

How to install local-llm-claude-code?

mjunaidca/mjs-agent-skills/local-llm-claude-code

$npx -y skills add mjunaidca/mjs-agent-skills --skill local-llm-claude-code

Installs into the current project.

Summary

Files · 1

View on GitHub

SKILL.md

1	# Local LLM → Claude Code
2
3	Connect Claude Code to a locally-served open model. Since **Ollama v0.14+ natively speaks the
4	Anthropic Messages API (`/v1/messages`), this needs no proxy** — just three env vars.
5
6	## The one thing to say up front (set expectations honestly)
7
8	Making this work and making it usable are different. Two independent thresholds must BOTH
9	be cleared, and cheap hardware usually misses both:
10
11	1. Capability — the model must emit valid structured tool calls, turn after turn.
12	Small models (≤1.7B) malform them (`args` object vs string, skipped tools). Fixed by a
13	bigger model (≥14B, ideally a 30B-class MoE), NOT by faster hardware.
14	2. Throughput — the box must chew Claude Code's ~18,000-token system prompt every turn
15	fast enough to beat its request timeout. CPU-only = minutes/turn. Fixed by a GPU, NOT
16	by a smarter model.
17
18	Say this to the user before installing, then read `references/hardware-sizing.md` and run
19	`scripts/check-hardware.sh` to give them a specific, honest verdict for their box.
20
21	## Workflow
22
23	Run these in order. Each script is idempotent and needs no root (user-space install).
24
25	### 1. Reality-check the hardware
26	```bash
27	bash scripts/check-hardware.sh
28	```
29	Reports CPU/RAM/GPU/disk and recommends a model tier (or warns it'll only be a learning rig).
30	Details + the sizing table: `references/hardware-sizing.md`.
31
32	### 2. Install Ollama (lean, user-space)
33	```bash
34	bash scripts/install-ollama.sh
35	```
36	Streams the release tarball through `zstd` and excludes the CUDA/ROCm/MLX GPU libraries,
37	so a CPU box installs in ~112 MB instead of ~3 GB. Starts `ollama serve` on `:11434`.
38	(On a GPU box, drop the `--exclude` flags — see the script's header.)
39
40	### 3. Pull + tune a model
41	```bash
42	bash scripts/make-model.sh <base-model> <derived-name>
43	# e.g. bash scripts/make-model.sh qwen3:14b qwen3-cc
44	```
45	Pulls the base model and builds a derived model with a Modelfile that sets
46	`num_ctx` large enough for Claude Code's ~18K prompt (default 32768). This is the #1 cause
47	of "local Claude Code is broken" — the Ollama default of 4096 silently truncates the prompt.
48
49	### 4. Launch Claude Code against it
50	```bash
51	bash scripts/launch.sh <derived-name> # e.g. qwen3-cc
52	```
53	Exports the three env vars and runs `claude --model <name>`. The wiring:
54	```bash
55	export ANTHROPIC_BASE_URL="http://localhost:11434"
56	export ANTHROPIC_AUTH_TOKEN="ollama" # required but ignored locally
57	export ANTHROPIC_API_KEY="" # MUST be empty so the token is used
58	```
59	On slow (CPU) boxes the launcher also sets `API_TIMEOUT_MS=900000` so turns don't 500 mid-prefill.
60
61	### 5. Verify end-to-end
62	```bash
63	bash scripts/verify.sh <derived-name>
64	```
65	Tests, in order: native `/v1/messages` chat → a real tool call → `count_tokens` (a 404 here
66	is FINE on v0.14+, not a hang) → measures prompt-eval tok/s so you can predict per-turn latency.
67
68	## Hard-won gotchas (all verified the hard way)
69
70	- No proxy needed. Ollama v0.14+ serves the Anthropic API natively. Do NOT install LiteLLM
71	or claude-code-router for Ollama. (vLLM/LM Studio DO need a proxy — see `references/other-backends.md`.)
72	- `num_ctx` ≥ 32768. Claude Code's system prompt is ~18K tokens. The 4096 default breaks it.
73	- Reasoning models (Qwen3, DeepSeek-R1) think by default and it's slow. You CANNOT disable
74	thinking through the Anthropic `/v1/messages` path (no `think` field passes; there's no
75	`PARAMETER think` in Modelfiles; template edits don't change server-side parse). For Claude
76	Code, either accept the overhead or choose a non-reasoning model. For YOUR OWN agents, call
77	Ollama's native `/api/chat` with `"think": false` — that DOES disable it (big speed win).
78	- `count_tokens` returns 404 on v0.14+ — harmless; Claude Code falls back to local estimation.
79	A hang (not a 404) means your Ollama is too old — upgrade.
80	- Prompt cache helps a lot. Turn 2+ reuses the cached ~18K system-prompt prefix, so only the
81	first turn pays full prefill. Any context growth past the cached prefix re-triggers eval.
82	- Install lean or run out of disk. The full tarball unpacks to ~3 GB (GPU libs); stripping
83	them → ~112 MB. Critical on small VPSs.
84
85	## When the us

People who liked this also viewed

6 picks

Type