README down to what a user needs; the rest into docs/
The README had grown to 1,010 lines of user docs and design notes in one file,
and had gone stale: it still showed the old numbered menu, set up only three
providers, and listed preflight checks and a file layout that predate the last
three releases. It now carries install, first run, the full command
reference, the providers at a glance, troubleshooting and a docs index.
docs/:
- providers.md presets, defaults, the model cache, context windows, each
provider (OpenRouter's cost guard and guardrail check are
written up for the first time), adding a provider
- live-sessions.md what a switch does to running sessions, and repair
- design.md why settings.json, why keys stay out of it (and the vault
per platform), the preflight checks as they are now
- bar-widget.md the widget as it is now: providers from health.json, every
server provider's settings, the restart after an upgrade
- architecture.md the pieces, every file on disk and who writes it, the
contracts between them, where to change what
- development.md running the tests, the conventions the code follows,
working on the widget, releasing
CONTRIBUTING.md points at it.
Also:
- The per-project session listing used awk, which the CLI avoids because it
is missing from minimal images; it uses the script's own TSV helpers now,
and tests/static.sh fails on any awk in the CLI.
- tests/static.sh checks every relative Markdown link and #anchor.
- A unit test pins the managed env keys between cm-json.py and
claude-mode.ps1, which only a comment kept in step before.
- test_sessions covers the per-project listing, which nothing ran.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
smoido
@@ -0,0 +1,305 @@
|
||||
# Providers and presets
|
||||
|
||||
A **provider** is where Claude Code's requests go: OpenRouter, Z.AI, an LM Studio
|
||||
or Ollama server, or a custom endpoint. Anthropic is not one — it is the native
|
||||
login, and switching to it removes every trace of the others. A **preset** is a
|
||||
named mapping of Claude Code's four model tiers (opus, sonnet, haiku, fable) onto
|
||||
one provider's model ids, plus the endpoint and how to authenticate to it.
|
||||
|
||||
- [Presets](#presets)
|
||||
- [Context windows and early auto-compaction](#context-windows-and-early-auto-compaction)
|
||||
- [OpenRouter](#openrouter) · [Z.AI](#zai) · [LM Studio](#lm-studio) · [Ollama](#ollama) · [Custom endpoints](#custom-endpoints)
|
||||
- [Adding a provider](#adding-a-provider)
|
||||
|
||||
## Presets
|
||||
|
||||
One ships per provider, so `claude-mode <provider>` is never ambiguous:
|
||||
|
||||
| preset | provider | opus | sonnet | haiku | fable |
|
||||
|---|---|---|---|---|---|
|
||||
| `default` | openrouter | `deepseek/deepseek-v4.1-flash` | `z-ai/glm-5.3-flash` | `openrouter/free` | `z-ai/glm-5.3` |
|
||||
| `zai` | zai | `glm-5.3` | `glm-5.3` | `glm-4.7` | `glm-5.3` |
|
||||
| `lmstudio` | lmstudio | whatever setup finds on your server (all tiers) | | | |
|
||||
| `ollama` | ollama | `qwen3-coder` until setup picks from your server (all tiers) | | | |
|
||||
| `custom` | custom | empty until setup | | | |
|
||||
|
||||
Presets are plain JSON in `~/.claude-mode/presets/`, shared verbatim between the
|
||||
Windows and POSIX builds. A preset declares its `provider`, and
|
||||
`claude-mode lmstudio default` is rejected rather than silently pointing a local
|
||||
URL at remote model ids. Build more with `preset new` — or from the bar panel.
|
||||
|
||||
To route Anthropic's own models through OpenRouter, copy a preset and repoint its
|
||||
tiers at `anthropic/claude-opus-5` and friends — and add `"allowAnthropicModels":
|
||||
true`, because the [cost guard](#openrouter) refuses them otherwise.
|
||||
|
||||
### Which preset a bare `claude-mode <provider>` picks
|
||||
|
||||
Never "most recently used". Out of the box it is a fixed name — `default`, `zai`,
|
||||
`lmstudio`, `ollama`, `custom` — and it can be changed:
|
||||
|
||||
```bash
|
||||
claude-mode preset default # what each provider picks, and why
|
||||
claude-mode preset default openrouter cheap # `claude-mode openrouter` now means `cheap`
|
||||
claude-mode preset default openrouter --clear # back to the built-in name
|
||||
```
|
||||
|
||||
The choice lives in `~/.claude-mode/defaults.json`. Renaming the chosen preset
|
||||
moves the choice along and deleting it clears it; a choice whose file has gone
|
||||
some other way falls back to the built-in name, and if that is gone too, to the
|
||||
provider's first preset by name. The Windows build does not read this file yet
|
||||
and keeps the built-in names.
|
||||
|
||||
### Editing the preset in use
|
||||
|
||||
It is re-applied straight away. For `preset set` and `preset all` that happens
|
||||
**without** asking about running sessions: what makes a switch dangerous to them
|
||||
is their endpoint or key moving, and a tier edit moves neither. `preset url` and
|
||||
`preset auth` do move them, so those still ask. If the re-apply fails (a
|
||||
preflight refusal, say), the edit is still saved, and the last line says so and
|
||||
names the command that finishes the job.
|
||||
|
||||
### The model cache
|
||||
|
||||
Every catalogue fetch — `models`, `doctor`, `setup`, the menu's picker — leaves a
|
||||
copy in `~/.claude-mode/models-cache.json`: one entry per provider, each with its
|
||||
own timestamp. A failed fetch keeps the previous list and marks it failed, and a
|
||||
server provider's list is tied to the server it came from. Model ids, context
|
||||
lengths and prices only; no key or key name is ever written there. The bar
|
||||
panel's model picker reads it, so the panel never touches the network itself.
|
||||
|
||||
## Context windows and early auto-compaction
|
||||
|
||||
**Symptom:** on a gateway, the session starts auto-compacting almost immediately,
|
||||
although every model involved has a huge context window.
|
||||
|
||||
**Cause:** behind a custom `ANTHROPIC_BASE_URL`, Claude Code cannot resolve a
|
||||
third-party model id like `deepseek/deepseek-v4-flash` to a context length. It
|
||||
falls back to a conservative default and compacts against *that*. Z.AI's own docs
|
||||
work around the same thing with `CLAUDE_CODE_AUTO_COMPACT_WINDOW=1000000`.
|
||||
|
||||
**Fix:** every preset carries `contextTokens`, which writes both knobs:
|
||||
|
||||
```
|
||||
CLAUDE_CODE_MAX_CONTEXT_TOKENS = <contextTokens>
|
||||
CLAUDE_CODE_AUTO_COMPACT_WINDOW = <contextTokens>
|
||||
```
|
||||
|
||||
| preset | contextTokens |
|
||||
|---|---|
|
||||
| `default`, `zai` | 1,000,000 |
|
||||
| `lmstudio` | 262,144 |
|
||||
| `ollama` | 65,536 — and see [Ollama](#the-context-window-is-set-on-the-server) |
|
||||
| `custom` | 200,000 |
|
||||
|
||||
`doctor` checks the declared window against each tier's actual model window and
|
||||
names any tier that falls short — `default` maps haiku to `openrouter/free`
|
||||
(200k), which it flags as harmless, since haiku only runs short background tasks.
|
||||
Adjust it in the preset file and run `claude-mode doctor` again.
|
||||
|
||||
## OpenRouter
|
||||
|
||||
A remote gateway to most vendors' models, paid per token. Setup asks for your key
|
||||
(`sk-or-…`) and picks a model per tier from OpenRouter's live catalogue, which
|
||||
shows each model's context length and price.
|
||||
|
||||
**The cost guard.** Gateways resell Anthropic's models at full list price, with no
|
||||
subscription discount, so routing a tier at one is almost never what was meant. A
|
||||
switch refuses any preset that does, naming the tier, unless the preset says
|
||||
`"allowAnthropicModels": true`. The same guard applies to every gateway provider.
|
||||
|
||||
**The guardrail check.** The cost guard only sees what a preset asks for; Claude
|
||||
Code can still request an Anthropic model on its own (a cached choice, a
|
||||
`/model` command). OpenRouter can block that server-side with a *guardrail* on
|
||||
the key, and `doctor` — and every switch to OpenRouter — checks whether one is in
|
||||
place by asking for the cheapest Anthropic model: refused means the guardrail is
|
||||
**active**; answered means it is **NOT SET**, with the steps to set one
|
||||
(openrouter.ai → Guardrails → exclude anthropic models). `doctor` also shows the
|
||||
key's spend and limit.
|
||||
|
||||
## Z.AI
|
||||
|
||||
Replaces `npx @z_ai/coding-helper`, and does something it does not: maps a
|
||||
distinct model to each tier instead of one model everywhere. Per
|
||||
[Z.AI's Claude Code docs](https://docs.z.ai/devpack/tool/claude):
|
||||
|
||||
| setting | value |
|
||||
|---|---|
|
||||
| `ANTHROPIC_BASE_URL` | `https://api.z.ai/api/anthropic` |
|
||||
| auth | your Z.AI key, from the vault through `apiKeyHelper` |
|
||||
| `API_TIMEOUT_MS` | `3000000` |
|
||||
| `CLAUDE_CODE_AUTO_COMPACT_WINDOW` | `1000000` |
|
||||
| `CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC` | `1` |
|
||||
|
||||
Z.AI publishes no model catalogue endpoint, so its list is kept in
|
||||
`providers.json` (from those docs), and a model id missing from it is reported
|
||||
as undocumented rather than wrong. Get a key at
|
||||
<https://z.ai/manage-apikey/apikey-list>; `doctor` sends a 1-token request to
|
||||
prove it works.
|
||||
|
||||
## LM Studio
|
||||
|
||||
Per [LM Studio's docs](https://lmstudio.ai/docs/integrations/claude-code): base
|
||||
URL `http://127.0.0.1:1234` (**base only** — Claude Code appends `/v1/messages`),
|
||||
token `lmstudio`, plus `CLAUDE_CODE_ATTRIBUTION_HEADER=0`. Setup maps one model
|
||||
across every tier, since a local server holds one in memory at a time and
|
||||
per-tier mapping would pay the load cost on every tier change.
|
||||
|
||||
### Model ids are not what the UI shows
|
||||
|
||||
LM Studio's `/v1/models` lists only **loaded** instances, under their display
|
||||
alias. `kat-coder-v2.5` is such an alias — once the model idle-unloads it
|
||||
vanishes, and a request by that name returns `400 No models loaded`. The
|
||||
JIT-loadable id is the model key, `kwaipilot_kat-coder-v2.5-dev`. claude-mode reads
|
||||
`/api/v0/models` instead, which lists every installed model with its load state,
|
||||
so `models` and `doctor` show ids that work.
|
||||
|
||||
### The `[Server Error] ... Unable to generate parser for this template` spam
|
||||
|
||||
Some GGUF chat templates hard-assert message ordering:
|
||||
|
||||
```jinja
|
||||
{%- if message.role == "system" %}
|
||||
{%- if not loop.first %}
|
||||
{{- raise_exception('System message must be at the beginning.') }}
|
||||
```
|
||||
|
||||
Runtimes that generate a tool-call parser probe the template with synthetic
|
||||
message sequences; the probes trip the assertion and the request dies. It is a
|
||||
model-template bug, reported against several models
|
||||
([LM Studio #1999](https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/1999),
|
||||
[llama.cpp #20733](https://github.com/ggml-org/llama.cpp/issues/20733)). The
|
||||
Windows build's `doctor` and `models` flag affected models as `TEMPLATE RISK`.
|
||||
**The fix is a model without the flag** — `claude-mode setup lmstudio` and pick
|
||||
another. Whether a flagged template actually trips depends on the parser strategy
|
||||
the runtime picks, so a flagged model may work fine for you.
|
||||
|
||||
Also: use a model with more than 25k of context (`doctor` warns below that), and
|
||||
the model must be **installed** — JIT loading handles "not loaded" fine.
|
||||
|
||||
### LM Studio somewhere other than this machine
|
||||
|
||||
Loopback is a default, not a constraint:
|
||||
|
||||
```bash
|
||||
claude-mode preset url lmstudio http://192.168.1.40:1234 # another box on the LAN
|
||||
claude-mode preset url lmstudio https://lms.example.net # through a tunnel or proxy
|
||||
claude-mode preset auth lmstudio key lmstudio # that server wants an API key
|
||||
claude-mode set-key lmstudio # store it (vault, not settings.json)
|
||||
claude-mode preset auth lmstudio none # back to the open-server default
|
||||
```
|
||||
|
||||
`auth none` writes the inline placeholder token, which is not a secret and is what
|
||||
an unauthenticated server expects. `auth key` moves it to the vault like every
|
||||
other credential — a real key on a public address is a real key.
|
||||
|
||||
## Ollama
|
||||
|
||||
Ollama serves Anthropic's Messages API itself, at `/v1/messages`, so nothing sits
|
||||
in between. `claude-mode setup ollama` asks where the server is (it ships on
|
||||
`http://127.0.0.1:11434`), whether it needs a key, and picks one model for every
|
||||
tier from what the server has pulled. The token is a placeholder, `ollama`, which
|
||||
Ollama requires but does not check.
|
||||
|
||||
### The context window is set on the server
|
||||
|
||||
This is the part that bites. Ollama sizes the context on the server, not per
|
||||
request: **4096 tokens** unless `ollama serve` runs with `OLLAMA_CONTEXT_LENGTH`,
|
||||
and anything past it is cut off without an error. Claude Code's system prompt
|
||||
alone is most of that. Ollama recommends 64k or more for Claude Code, and the
|
||||
shipped preset declares 65,536 — but `contextTokens` only tells Claude Code what
|
||||
to expect. It cannot change what the server does.
|
||||
|
||||
```bash
|
||||
OLLAMA_CONTEXT_LENGTH=65536 ollama serve
|
||||
# or, for the systemd service: systemctl edit ollama
|
||||
# [Service]
|
||||
# Environment=OLLAMA_CONTEXT_LENGTH=65536
|
||||
```
|
||||
|
||||
`claude-mode doctor` reads what each loaded model is actually running with
|
||||
(`/api/ps`) and the model's own maximum (`/api/show`), and says so when either is
|
||||
below the preset. When nothing is loaded it cannot see the server's setting, and
|
||||
says that instead of guessing.
|
||||
|
||||
Also worth knowing:
|
||||
|
||||
- A bare name like `qwen3-coder` is `qwen3-coder:latest` to Ollama; both work.
|
||||
- Ollama does not implement `count_tokens` or prompt caching, so long sessions
|
||||
redo more work than they would on a hosted provider. Nothing breaks.
|
||||
- To reach it from another machine, serve it with `OLLAMA_HOST=0.0.0.0` and point
|
||||
the preset there: `claude-mode preset url ollama http://192.168.1.40:11434`.
|
||||
|
||||
## Custom endpoints
|
||||
|
||||
For anything else that speaks Anthropic's Messages API: a LiteLLM or Vercel
|
||||
gateway, vLLM, llama.cpp's server, a company proxy. The `custom` preset ships
|
||||
with no address — there is no sensible one to guess — and a switch to it is
|
||||
refused until it has one.
|
||||
|
||||
```bash
|
||||
claude-mode setup custom # asks for all of the below
|
||||
# or by hand:
|
||||
claude-mode preset url custom https://llm.example.com
|
||||
claude-mode preset auth custom key custom # the key lives in the vault as 'custom'
|
||||
claude-mode set-key custom
|
||||
claude-mode preset set custom opus <model-id> # ...and each other tier
|
||||
```
|
||||
|
||||
Several endpoints are several presets, each with its own address and key name:
|
||||
`claude-mode preset new work --provider custom --blank`, then
|
||||
`claude-mode preset auth work key work`.
|
||||
|
||||
The model list comes from `/v1/models` when the endpoint has one (OpenAI's shape
|
||||
or Anthropic's). Plenty of proxies serve Messages and nothing else, and that is
|
||||
fine: the check before a switch only wants something to answer, and model ids can
|
||||
be typed by hand. The cost guard still applies — a custom endpoint in front of
|
||||
Anthropic's own models needs `"allowAnthropicModels": true`, which keeps it
|
||||
deliberate.
|
||||
|
||||
## Adding a provider
|
||||
|
||||
Every gateway provider is an entry in [`providers.json`](../providers.json),
|
||||
installed next to the presets and read by the POSIX CLI, the Windows build and
|
||||
the bar widget (through `health.json`). An entry holds:
|
||||
|
||||
| field | what it says |
|
||||
|---|---|
|
||||
| `id`, `aliases`, `title`, `label`, `blurb`, `color` | names, and how the CLI, the menu and the panel show it |
|
||||
| `defaultPreset`, `preset` | the preset `claude-mode <id>` picks, and the template for a blank one (URL, auth, context, extra env) |
|
||||
| `server` | whether the address is editable, how it is probed before a switch (`always`; `lenient`, for proxies; `local`, only when it points at this machine), the paths probed, setup hints |
|
||||
| `catalogue.kind` | how its model list is read: `openrouter`, `lmstudio`, `ollama`, `openai` (`/v1/models`) or `static` (a list kept in the entry) |
|
||||
| `setup` | whether a key is `required` or `optional`, where to get one, and `per-tier` or `one-for-all` model picking |
|
||||
| `doctor` | which checks apply: `catalogue-models`, `openrouter-key`, `guardrail`, `message-check`, `ollama-context`, `lmstudio-templates` (Windows) |
|
||||
| `logo`, `logoScale` | a single-path 24×24 SVG mark, and its optical size correction ([bar widget](bar-widget.md#the-marks)) |
|
||||
|
||||
A provider that reuses those kinds is an entry and a default preset in
|
||||
`presets/`, and **no code**. A hosted Anthropic-compatible coding plan is shaped
|
||||
like `zai`:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "example", "aliases": [], "title": "Example",
|
||||
"label": "Example - hosted coding plan",
|
||||
"blurb": "Example's coding plan on its Anthropic endpoint.",
|
||||
"color": "gray", "defaultPreset": "example",
|
||||
"preset": {"baseUrl": "https://api.example.com/anthropic",
|
||||
"auth": {"mode": "vault", "keyRef": "example"},
|
||||
"contextTokens": 200000, "gatewayModelDiscovery": false},
|
||||
"server": {"editable": false, "probe": "local"},
|
||||
"catalogue": {"kind": "openai"},
|
||||
"setup": {"key": "required", "models": "per-tier", "keyUrl": "https://example.com/keys"},
|
||||
"doctor": ["catalogue-models", "message-check"],
|
||||
"logo": "M0 0h24v24H0z", "logoScale": 1.0, "glyph": ""
|
||||
}
|
||||
```
|
||||
|
||||
…plus `presets/example.json` with `"provider": "example"` and
|
||||
`"configured": false`. `scripts/test.sh` checks the entry against the kinds the
|
||||
code implements, that its default preset exists, and that its id cannot shadow a
|
||||
command.
|
||||
|
||||
Only a genuinely new *kind* — a catalogue format none of the parsers read, a probe
|
||||
that behaves differently — needs code: a parser in `linux/cm-json.py`, a branch
|
||||
in `provider_catalogue` (bash) and `Get-ProviderCatalogue` (`claude-mode.ps1`),
|
||||
and the new value added to the check in `tests/static.sh`. The bar widget picks
|
||||
new providers up from `health.json`, with no QML change and no shell restart.
|
||||
Reference in New Issue
Block a user