Skip to content

Claude Code in Your Own Web App · Part 15 of 16 in the series

Part 15: A Local Model as the Brain

Let a site's Claude Code think with a model on your own machine: Ollama answers Anthropic's API, a few environment variables point the session at it, and the rest of Sitesmith stays as it is. With an honest look at what a small model on a CPU can do.

Quest objectives

  • Run Ollama in Docker with a context long enough for Claude Code
  • Point a Claude Code session at Ollama's Anthropic-compatible API, and know what each variable does
  • Give every site its own brain, with a new conversation when it changes
  • Pick the model in the page from the list Ollama has
  • Know the timeouts a slow model runs into, and what to expect from a model on a CPU

For fourteen parts, every site's Claude Code has thought with Claude, on your plan. That is the best brain Sitesmith can have, but it is not the only one Claude Code can talk to. Claude Code sends its requests to Anthropic's Messages API, and the address of that API is a setting. Ollama, the program that runs language models on your own machine, answers the same API. Point a session at it, and Claude Code runs as before, with its tools, its hooks and its permission prompts, while the thinking happens on your machine.

This part adds that as a choice per site: a picker in the site's header with "Claude (your plan)" and every model Ollama has. Sitesmith itself barely changes: a file that remembers the choice, a few environment variables for the session, one more option for Claude Code.

One thing up front, so you know how much time to give this part. On the machine this series was tested on (16 CPU threads, no graphics card for the model), the plumbing worked: the request reached the model, and its answer came back into the page. The model itself was another story: it spent 26 minutes on Claude Code's first request, most of them just reading it, and then answered something nobody had asked, without touching a file. The last sections are about why, and what to try instead.

The same Claude Code, the same hooks, the same page. Only the brain behind it moves onto your own machine.
The same Claude Code, the same hooks, the same page. Only the brain behind it moves onto your own machine.

How Claude Code talks to another model

Claude Code is a program on your machine; the model is somewhere else. Between them runs the Messages API: Claude Code sends the conversation, its system prompt and the descriptions of its tools to /v1/messages, and the model answers with text and tool calls. The tools themselves Claude Code runs on its own: it reads the file, asks for permission, runs the hooks. So everything Sitesmith is built on, the hooks, the questions, the permission prompts, plan mode, belongs to Claude Code, not to the model.

Since version 0.14, Ollama answers that API too. It translates the requests for its models and their answers back, thinking and tool calls included. Claude Code does not need to know that it talks to something else. It needs a different address, and a handful of settings that would otherwise point at Anthropic's models.

Ollama in Docker

Ollama runs without Docker too, with installers for Linux, macOS and Windows. Docker keeps it in a box you can throw away, and it is how it was tested here:

~/sitesmith
$ docker run -d --name ollama -p 127.0.0.1:11434:11434 -e OLLAMA_CONTEXT_LENGTH=32768 -e OLLAMA_FLASH_ATTENTION=1 -e OLLAMA_KV_CACHE_TYPE=q8_0 -v ollama:/root/.ollama ollama/ollama42e31ac0de4a…$ curl -s localhost:11434/api/version{"version":"0.40.16"}$ docker exec ollama ollama pull qwen3:1.7b# …success$ curl -s localhost:11434/api/show -d '{"model":"qwen3:1.7b"}'# … (only the capabilities)["completion","tools","thinking"]

The options of docker run, one by one:

  • -p 127.0.0.1:11434:11434: the port on this machine only. Ollama has no login, and whoever reaches the port can use your models.

  • OLLAMA_CONTEXT_LENGTH=32768: how many tokens a model may hold at once. More on this number right below; it is the setting that matters most.

  • OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0: the memory for the context, the KV cache, is kept at 8 bits instead of 16, about half the size. Ollama does that only with flash attention on. A long context costs memory, and this halves the bill.

  • -v ollama:/root/.ollama: the models live in a Docker volume, so they survive a new container.

/api/version shows that Ollama answers; any version from 0.14 on has the Anthropic API, and this was 0.40.16. ollama pull, run inside the container, downloads the model. qwen3:1.7b is a small one, with 1.7 billion parameters, picked because this machine had about 6 GB of memory free. /api/show describes a model; of its long answer only the capabilities are shown here. The one that counts is tools. Claude Code does all its work through tools, reading a file, writing one, running a command, and a model without tool support can only talk.

The context length trap

A language model reads the whole conversation every time it answers, and the context length is how much of it fits. Ollama picks a default from the memory of your graphics card: below 24 GB of VRAM, and so on a machine without one, that is 4k tokens.

Claude Code's first request in the test was 19,602 tokens, almost all of it before a word of the request itself: its system prompt and the descriptions of its tools. With the default, four fifths of it do not fit. Ollama does not turn such a request away; it shortens it to what fits, and the model goes to work with only a part of the instructions and tool descriptions Claude Code sent. Claude Code does not hear about it, and in the page you only see a model that behaves strangely.

Ollama's documentation asks for at least 64k tokens for agents and coding tools, and that is what to use if your machine has the memory: the context needs memory on top of the model, and more the longer it is. This machine had room for 32k, which is enough for a start. The first request fits, with some 13,000 tokens to spare, but a conversation fills that quickly: every file Claude Code reads goes into it. With the memory, write 65536 in the docker run above.

Ollama's Anthropic endpoint, tried by hand:

~/sitesmith
$ curl -s localhost:11434/v1/messages -H 'content-type: application/json' -H 'x-api-key: ollama' -H 'anthropic-version: 2023-06-01' -d '{"model":"qwen3:1.7b","max_tokens":200,"messages":[{"role":"user","content":"Say hello in five words."}]}'{"id":"msg_6673e250b63dabbbc663a282","type":"message","role":"assistant","model":"qwen3:1.7b","content":[{"type":"thinking","thinking":"Okay, the user asked to say hello in five words. …

That took 7.9 seconds, and the answer began with a thinking block: qwen3 thinks before it answers, and Ollama hands the thinking over the way Claude's arrives. Mind the address. The API sits under /v1/messages, and Claude Code adds that part itself, so what Claude Code gets is the root, http://127.0.0.1:11434, without /v1.

A brain per site

The choice belongs to a site: one site may be built with Claude, another with a local model you want to try. First the types, at the top of shared/types.ts. A Brain is either your plan or a local model by its name. A site carries its brain, and LocalModels answers the question which models there are (Config stays as it is):

shared/types.ts
/** What the server and the page both know about. */

/** What Claude Code thinks with: your Claude plan, or a model on your own machine (through Ollama). */
export type Brain = { kind: 'plan' } | { kind: 'local'; model: string };

/** A site: one folder in sites/, which Claude Code builds. */
export interface Site {
  /** the folder's name: also the site's address in the page and in the preview */
  name: string;
  title: string;
  description: string;
  created: number;
  /** whether it has an index.html yet */
  built: boolean;
  brain: Brain;
}

export interface NewSite {
  name: string;
  title: string;
  description: string;
}

export interface Config {
  /** the preview server's port: the sites are shown from there, never from the page's own address */
  previewPort: number;
  /** whether tmux was found: without it there are no sessions */
  terminals: boolean;
  /** whether node-pty was found: without it the page can't show the sessions */
  pty: boolean;
}

/** The models Ollama has, or why there are none. */
export interface LocalModels {
  models: string[];
  error?: string;
}

The choice lives in the site's folder, next to site.json: .sitesmith/brain.json. In server/sites.ts, add Brain to the type import and the file's name below the others:

server/sites.ts
/** Sitesmith's own files in a site's folder. */
export const OWN = '.sitesmith';
const META = `${OWN}/site.json`;
const BRAIN = `${OWN}/brain.json`;

Then replace read, and add three methods below it:

server/sites.ts
read(name: string): Site | null {
  if (!NAME.test(name)) return null;
  try {
    const meta = JSON.parse(fs.readFileSync(path.join(this.dir(name), META), 'utf8')) as Omit<Site, 'name' | 'built' | 'brain'>;
    return { ...meta, name, built: fs.existsSync(path.join(this.dir(name), 'index.html')), brain: this.brain(name) };
  } catch { return null; }
}

/** What the site's Claude Code thinks with: your plan, unless a local model was chosen. */
brain(name: string): Brain {
  try {
    const b = JSON.parse(fs.readFileSync(path.join(this.dir(name), BRAIN), 'utf8')) as Brain;
    if (b.kind === 'local' && typeof b.model === 'string' && b.model) return { kind: 'local', model: b.model };
  } catch { /* none chosen */ }
  return { kind: 'plan' };
}

/** A brain from what the page sent, checked: a local model needs a plain name. */
brainFrom(input: Partial<Brain>): Brain {
  if (input.kind !== 'local') return { kind: 'plan' };
  const model = String((input as { model?: string }).model ?? '').trim();
  if (!/^[\w.:/-]{1,100}$/.test(model)) throw new HttpError(400, 'Pick a local model.');
  return { kind: 'local', model };
}

setBrain(name: string, brain: Brain) {
  fs.writeFileSync(path.join(this.dir(name), BRAIN), `${JSON.stringify(brain)}\n`);
}

brain reads the file and falls back to the plan for anything else: no file, a broken one, a model without a name. A site nobody chose for, which is every site so far, stays on your plan. brainFrom takes what the page sends and checks it, and a bad name is a 400. A model's name may only hold the characters Ollama's names have (letters, digits, _ . : / -), because it ends up as an argument of Claude Code and in its environment. setBrain only writes the file; the route below checks first and writes last. The folder watch from part 2 ignores .sitesmith/, so writing the file does not look like a change to the site.

A switch starts a new conversation

A session's brain is fixed when it starts: the variables and the --model option reach Claude Code when tmux starts it. So a switch has to end the session, and the next request starts a new one with the new brain. In server/agent.ts, the class gets a set of the sites whose next session starts afresh:

server/agent.ts
private sites = new Map<string, SiteAgent>();
private pending = new Map<string, Pending>();
/** sites whose next session starts a new conversation (their brain changed) */
private fresh = new Set<string>();

Add this method to the class, above stop:

server/agent.ts
/**
 * Another brain for the site: it takes effect in a new session, with a new conversation (a long
 * conversation of one model is a poor start for another). Not in the middle of a run.
 */
async brainChanged(t: Target) {
  if (this.active(t.name) || this.site(t.name).ask) throw new Error('Claude Code is working on this site. Switch when it is done.');
  this.fresh.add(t.name);
  await this.terminals.end(t);
}

And replace deliverNext:

server/agent.ts
private async deliverNext(name: string, s: SiteAgent) {
  const job = s.queue[0];
  if (!job) return;
  await this.terminals.poll();
  const term = this.terminals.get(name);
  // a conversation continues, unless the site's brain changed since
  const resume = this.terminals.hasConversation(job.target.dir) && !this.fresh.has(name);
  try {
    if (!term) await this.terminals.start(job.target, { prompt: COMMAND, resume, plan: job.plan });
    else if (!term.claude) await this.terminals.respawn(job.target, { prompt: COMMAND, resume, plan: job.plan });
    else {
      if (this.busy(name) || !(await this.terminals.atEmptyPrompt(job.target))) return;
      // it may have been taken while we looked
      if (s.queue[0] !== job) return;
      // plan mode for a plan first, and out of it for everything else
      if (!(await this.terminals.setMode(job.target, job.plan ? 'plan' : 'acceptEdits'))) {
        throw new Error(`Claude Code did not switch to ${job.plan ? 'plan mode' : 'accepting edits'}: look in the terminal.`);
      }
      await this.terminals.type(job.target, COMMAND);
    }
  } catch (err) {
    s.queue.shift();
    s.run = job;
    this.end(name, 'failed', (err as Error).message);
    return;
  }
  s.queue.shift();
  s.run = job;
  if (!term?.claude) this.fresh.delete(name);
  job.status = 'starting';
  job.started = job.sentAt = Date.now();
  this.changed(name);
}

Two lines are new in it. resume continued the folder's last conversation whenever there was one (part 5); after a switch it does not. And once a new session has started, by start or respawn and not by typing into a running one, the site leaves the set again, and the following requests continue the new conversation as before.

Why not continue the old conversation with the new model? Because it is a poor start for the other one. An afternoon's conversation with Claude can be longer than a local model's whole context, so it would not even fit. And the other way round, Claude would carry on from whatever the small model did. The real state of a site is its files anyway: a new conversation starts with the folder as it is, and with the CLAUDE.md from part 2, which tells any model what the folder is.

brainChanged refuses while Claude Code works on the site or waits for an answer in the page, since ending the session would cut a run off in the middle. If you start the session by hand from the header after a switch, "Start Claude Code" also gives you a new conversation. "Continue last conversation" takes up the old one on the new brain; that is then your decision.

Options and routes

In server/main.ts, below DATA, two options join the others, with a small function for Ollama's address:

server/main.ts
/** OLLAMA_HOST as Ollama reads it ("0.0.0.0", "box:11434", "https://box"), as an address to call */
function ollamaUrl(host = '127.0.0.1:11434') {
  const url = new URL(/^https?:\/\//.test(host) ? host : `http://${host}`);
  // the address Ollama listens on for everyone, called here
  if (url.hostname === '0.0.0.0') url.hostname = '127.0.0.1';
  if (!url.port && url.protocol === 'http:') url.port = '11434';
  return url.origin;
}
/** Ollama, for sites on a local model */
const OLLAMA = option('--ollama', ollamaUrl(process.env.OLLAMA_HOST || undefined));
/** the context length Ollama gives its models (OLLAMA_CONTEXT_LENGTH): Claude Code needs to know it */
const LOCAL_CONTEXT = Number(option('--local-context', '65536'));

const sites = new Sites(path.resolve(option('--sites', path.join(APP, 'sites'))));
const config: Config = { previewPort: PREVIEW_PORT, terminals: false, pty: false };

--ollama is Ollama's address as Claude Code should use it: the root, without /v1. Without the option, Sitesmith takes OLLAMA_HOST, the variable Ollama's own command line reads, or else 127.0.0.1:11434. ollamaUrl reads that variable the way Ollama does: it may be a bare host, a host with a port, or a full address. A missing scheme means http, a missing port 11434 (an https address keeps its own default port), and 0.0.0.0, which tells Ollama to listen on every network address, is reached at 127.0.0.1. --local-context is the context length you gave Ollama. Sitesmith does not read it from Ollama, so you pass the same number. Its default, 65,536, is the 64k the documentation asks for. Add Brain and LocalModels to the type import, and the two options to the usage line at the top of the file: [--ollama URL] [--local-context 65536].

The terminals learn both right after they are created, and target, which every route uses to find a site, now hands on its brain as well. localModels asks Ollama for its models (/api/tags, what ollama list shows), with three seconds of patience:

server/main.ts
const terminals = new Terminals(APP, broadcast);
terminals.setOllama(OLLAMA, LOCAL_CONTEXT);
await terminals.init();
config.terminals = terminals.available;
config.pty = terminals.canAttach;
const agent = new Agent(APP, DATA, terminals, broadcast, () => clients.size);

/** A site's name and folder: what the terminals need, after checking the site exists. */
function target(name: string): Target {
  const site = sites.get(name);
  return { name: site.name, dir: sites.dir(site.name), brain: site.brain };
}

/** The models Ollama has: its /api/tags. */
async function localModels(): Promise<LocalModels> {
  try {
    const res = await fetch(`${OLLAMA}/api/tags`, { signal: AbortSignal.timeout(3000) });
    const data = await res.json() as { models?: { name: string }[] };
    return { models: (data.models ?? []).map((m) => m.name).sort() };
  } catch {
    return { models: [], error: `Ollama does not answer at ${OLLAMA}.` };
  }
}

Two routes go into the list, after GET /api/sites/…:

server/main.ts
// what a site's Claude Code thinks with
['GET', /^\/api\/local-models$/, async (_req, res) => send(res, 200, await localModels())],
['PUT', /^\/api\/sites\/([^/]+)\/brain$/, async (req, res, m) => {
  const t = target(m[1]);
  const brain = sites.brainFrom(await readJson<Brain>(req));
  try { await agent.brainChanged(t); } catch (err) { throw new HttpError(409, (err as Error).message); }
  sites.setBrain(t.name, brain);
  broadcast({ type: 'sites', sites: sites.list() });
  send(res, 200, sites.get(t.name));
}],

The PUT route checks in an order that leaves nothing half done. First brainFrom: a bad model name is a 400 before anything happens. Then brainChanged: if Claude Code is busy, nothing changes and the page gets a 409 with the reason; otherwise the session ends. Only then is the file written, and every open page gets the new list of sites, with the brain in it.

The session's environment

Everything the local model needs reaches Claude Code through its session. In server/terminals.ts, add Brain to the type import, and give the target and the start options a brain:

server/terminals.ts
export interface Target { name: string; dir: string; brain?: Brain }
/** How a session starts: with a first message, continuing the last conversation, in plan mode. */
export interface StartOptions { prompt?: string; resume?: boolean; plan?: boolean; brain?: Brain }

The class keeps Ollama's address and the context length next to the hooks:

server/terminals.ts
private hooks: { settings: string; data: string } | null = null;
/** where Ollama listens, for sessions on a local model, and the context length it gives them */
private ollama = { url: 'http://127.0.0.1:11434', context: 65536 };
available = false;

Replace env, with setOllama above it:

server/terminals.ts
setOllama(url: string, context: number) {
  this.ollama = { url, context };
}

/**
 * The session's environment: which site its hooks report for and where they find the server;
 * on a local model also where Claude Code sends its requests instead of Anthropic's API. The
 * core of what `ollama launch claude` sets: every model name Claude Code uses (its aliases, the
 * subagents') means the local model, and no API key is sent.
 */
private env(t: Target) {
  const vars: Record<string, string> = {};
  if (this.hooks) Object.assign(vars, { SITESMITH_SITE: t.name, SITESMITH_DATA: this.hooks.data });
  if (t.brain?.kind === 'local') {
    const model = t.brain.model;
    Object.assign(vars, {
      ANTHROPIC_BASE_URL: this.ollama.url,
      ANTHROPIC_AUTH_TOKEN: 'ollama',
      ANTHROPIC_API_KEY: '',
      // the endpoint above wins over one in your settings files
      CLAUDE_CODE_PROVIDER_MANAGED_BY_HOST: '1',
      ANTHROPIC_DEFAULT_OPUS_MODEL: model,
      ANTHROPIC_DEFAULT_SONNET_MODEL: model,
      ANTHROPIC_DEFAULT_HAIKU_MODEL: model,
      CLAUDE_CODE_SUBAGENT_MODEL: model,
      // Claude Code knows the context of its own models only: it would assume 1M tokens and
      // compact the conversation far too late
      CLAUDE_CODE_MAX_CONTEXT_TOKENS: String(this.ollama.context),
      // Ollama has no server-side checks for auto mode
      CLAUDE_CODE_AUTO_MODE_SERVER: '0',
      // a model on a CPU may read for many minutes before its first word: an hour for the
      // answer to begin (instead of 10 minutes), and no 5-minute limit on a pause within it
      API_TIMEOUT_MS: '3600000',
      API_FORCE_IDLE_TIMEOUT: '0',
    });
  }
  return Object.entries(vars).flatMap(([k, v]) => ['-e', `${k}=${v}`]);
}

Until now env gave the session the two variables of the hooks (part 6). Now it collects them in an object, adds the local model's when the site has one, and turns them all into tmux's -e NAME=value pairs. In part 3, tmux.conf took ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN and ANTHROPIC_API_KEY out of every session, so that nothing would send your sessions elsewhere by accident. Here they come back, on purpose, for one session. One by one:

  • ANTHROPIC_BASE_URL: where Claude Code sends its requests. Ollama's root address; Claude Code adds /v1/messages.

  • ANTHROPIC_AUTH_TOKEN=ollama: a token, which Claude Code sends as Authorization: Bearer ollama. Ollama on your machine does not check it, but with a token set, Claude Code works with that instead of your Claude login.

  • ANTHROPIC_API_KEY, empty: no API key, wherever one might come from. Ollama's documentation sets it empty the same way.

  • CLAUDE_CODE_PROVIDER_MANAGED_BY_HOST=1: says that the program which started Claude Code decides where its requests go. Claude Code then ignores addresses and credentials in settings files, so an env block in your ~/.claude/settings.json cannot send this session anywhere else. Without it, settings files win over the environment, as Ollama's documentation warns.

  • ANTHROPIC_DEFAULT_OPUS_MODEL, …_SONNET_MODEL, …_HAIKU_MODEL: what the aliases opus, sonnet and haiku stand for. Claude Code uses more than one model: Haiku does work in the background, and the aliases are what /model offers. Each would ask Ollama for a Claude model it does not have. Now they all mean the local model.

  • CLAUDE_CODE_SUBAGENT_MODEL: the model of subagents, the helpers Claude Code starts for a side task. The same local model.

  • CLAUDE_CODE_MAX_CONTEXT_TOKENS: the context Claude Code may count on. It knows the windows of its own models only, and for this one it warned: Model "qwen3:1.7b" isn't recognized · context assumed to be 1m tokens. Claude Code compacts a conversation, summarizes it, when it nears the window. Counting on a million tokens, it would not do that until long after Ollama had started dropping what does not fit. With the real number it compacts in time. That is why --local-context must match OLLAMA_CONTEXT_LENGTH.

  • CLAUDE_CODE_AUTO_MODE_SERVER=0: in auto mode, where Claude Code offers it, a check decides which actions go ahead without asking. On Anthropic's API, Claude Code has the server review them. Ollama has no such review, so with 0 Claude Code sends its own check requests, which now go to the local model like everything else. Sitesmith works in accept-edits mode, so this matters only if you switch to auto mode yourself.

  • API_TIMEOUT_MS and API_FORCE_IDLE_TIMEOUT: the timeouts, right below.

This is the core of what ollama launch claude sets as well; more on that command at the end.

The timeouts

Claude Code gives every request ten minutes for the answer to begin (strictly, for the response headers to arrive). For Claude that is generous. A model on a CPU first has to read the whole request, and Ollama sends nothing before it has. API_TIMEOUT_MS=3600000 gives it an hour. API_FORCE_IDLE_TIMEOUT=0 switches off a second limit: once an answer flows, Claude Code drops it after five minutes without a byte, on every endpoint that is not Anthropic's own, and a slow model can pause that long. Claude Code has further watchdogs for silent streams, which this variable does not touch. Both values are for slow machines, and the test below shows how slow that can be.

The model by name

Replace command, start and respawn:

server/terminals.ts
/**
 * What the session runs: Claude Code in a login shell, and when it exits, an interactive shell in
 * the folder, so the session stays open. The shell gets the program and its arguments as "$0"
 * and "$@", so no argument is ever parsed by the shell.
 */
private command(opts: StartOptions) {
  return [SHELL, '-lic', '"$0" "$@"; exec "$SHELL" -i', CLAUDE,
    // file edits in the site's folder go ahead, commands still ask; or plan mode for a plan first
    '--permission-mode', opts.plan ? 'plan' : 'acceptEdits',
    // a local model by its name in Ollama
    ...(opts.brain?.kind === 'local' ? ['--model', opts.brain.model] : []),
    // the hooks, on top of your own settings
    ...(this.hooks ? ['--settings', this.hooks.settings] : []),
    ...(opts.resume ? ['--continue'] : []),
    ...(opts.prompt ? [opts.prompt] : [])];
}

/** Starts Claude Code in a site's session: fresh, or continuing the folder's last conversation. */
async start(t: Target, opts: StartOptions = {}) {
  if (this.states.has(t.name)) return;
  await this.tmux('new-session', '-d', '-s', t.name, '-x', '120', '-y', '36', '-c', t.dir, ...this.env(t), ...this.command({ ...opts, brain: t.brain }))
    .catch((err: Error) => { if (!/duplicate session/.test(err.message)) throw err; });
  await this.poll();
}

/** Claude Code again, in place of the shell it left behind. */
async respawn(t: Target, opts: StartOptions = {}) {
  await this.tmux('respawn-pane', '-k', '-t', pane(t.name), '-c', t.dir, ...this.env(t), ...this.command({ ...opts, brain: t.brain }));
  await this.poll();
}

--model sets the main model by its name in Ollama. Claude Code does not know the name, and it does not have to: it sends it as it is, which is what the warning above was about. start and respawn hand the target's brain to command; the environment comes from env(t) as before.

The picker

Create src/brainPicker.ts:

src/brainPicker.ts
import { api } from './api.ts';
import { h } from './dom.ts';
import type { Brain, LocalModels, Site } from '../shared/types.ts';

/** the plan's option: a value no model's name can be */
const PLAN = '';

/**
 * What the site's Claude Code thinks with: your Claude plan, or one of the models Ollama has.
 * A switch ends the session; the next request starts it with the new brain and a new conversation.
 */
export class BrainPicker {
  readonly el: HTMLElement;
  private site: string;
  private select = h('select', { 'aria-label': 'What Claude Code thinks with' });
  private msg = h('span', { class: 'hint bad', 'aria-live': 'polite' });
  private brain: Brain = { kind: 'plan' };
  private models: string[] = [];

  constructor(site: string) {
    this.site = site;
    this.select.onchange = () => void this.choose();
    this.select.onfocus = () => void this.load();
    this.el = h('label', { class: 'brain' }, this.select, this.msg);
    void this.load();
  }

  /** Ollama's models, asked again whenever the list is opened (you may have pulled one since). */
  private async load() {
    const r = await api<LocalModels>('GET', '/api/local-models').catch(() => ({ models: [], error: 'No answer.' }) as LocalModels);
    this.select.title = r.error ?? '';
    // the same list again: leave the open list alone
    if (r.models.join() === this.models.join()) return;
    this.models = r.models;
    this.render();
  }

  update(site: Site | null) {
    if (!site || (JSON.stringify(site.brain) === JSON.stringify(this.brain) && this.select.options.length)) return;
    this.brain = site.brain;
    this.render();
  }

  private render() {
    const current = this.brain.kind === 'local' ? this.brain.model : PLAN;
    const models = [...new Set([...this.models, ...(this.brain.kind === 'local' ? [this.brain.model] : [])])];
    this.select.replaceChildren(
      h('option', { value: PLAN }, 'Claude (your plan)'),
      ...models.map((m) => h('option', { value: m }, `Local: ${m}`)));
    this.select.value = current;
  }

  private async choose() {
    const value = this.select.value;
    this.msg.textContent = '';
    if (!confirm('Switch the brain? Claude Code restarts with it, in a new conversation.')) { this.render(); return; }
    const brain: Brain = value === PLAN ? { kind: 'plan' } : { kind: 'local', model: value };
    try { await api('PUT', `/api/sites/${this.site}/brain`, brain); } catch (err) {
      this.msg.textContent = (err as Error).message;
      this.render();
    }
  }
}

A plain select: "Claude (your plan)" first, then "Local: …" for every model Ollama has. The plan's option has the empty value, which no model's name can be. The list is fetched again whenever the select gets the focus, so a model you have just pulled shows up without reloading the page; when the list is the same as before, the options are left alone, and so they are when the site's brain has not changed, so a list you have just opened is not rebuilt under your pointer. The site's current model stays in the list even when Ollama does not answer, so the select never shows a choice that is not true. When Ollama cannot be reached, the reason is in the select's tooltip.

A switch asks first, since it ends the session. If the server refuses, the message appears next to the select and the select goes back. On success the picker does nothing itself: the server sends every page the new list of sites, and update sets the select from it.

The list holds every model Ollama has, also ones that cannot use tools, an embedding model for instance. Pick one with tools among its capabilities.

In src/sitePage.ts, import BrainPicker, add a field private brain: BrainPicker; below session, and put the picker into the header, in front of the session's controls:

src/sitePage.ts
constructor(name: string, config: Config) {
  this.name = name;
  this.config = config;
  this.session = new SessionControls(name, config.terminals);
  this.brain = new BrainPicker(name);
  this.terminal = new TerminalView(name, config.pty);
  const attachments = new Attachments(name, this.previewUrl);
  this.request = new RequestBox(name, attachments);
  this.agent = new AgentPanel(name);
  this.tabs = {
    preview: h('button', { type: 'button', class: 'tab', role: 'tab', onclick: () => this.show('preview') }, 'Preview'),
    terminal: h('button', { type: 'button', class: 'tab', role: 'tab', onclick: () => this.show('terminal') }, 'Terminal'),
  };
  this.el = h('section', { class: 'site-page' },
    h('header', { class: 'site-head' },
      h('div', { class: 'site-head-text' }, this.heading, this.about),
      this.brain.el,
      this.session.el,
      h('a', { class: 'button small', href: this.previewUrl, target: '_blank', rel: 'noopener' }, 'Open in a new tab')),
    h('div', { class: 'site-body' },
      h('div', { class: 'work' }, this.request.el, this.agent.el),
      h('div', { class: 'view' },
        h('div', { class: 'tabs', role: 'tablist' }, this.tabs.preview, this.tabs.terminal),
        this.preview, this.terminal.el)));
  this.show('preview');
  dropTarget(this.el, (files) => void attachments.add(files));
}

And in update, one line at the end:

src/sitePage.ts
/** The list of sites changed (or arrived). */
update(sites: Site[]) {
  const site = sites.find((s) => s.name === this.name) ?? null;
  const wasBuilt = this.site?.built;
  this.site = site;
  this.heading.textContent = site?.title ?? this.name;
  this.about.textContent = site ? site.description : 'There is no site of this name.';
  this.frame.hidden = !site?.built;
  this.empty.hidden = Boolean(site?.built);
  if (site?.built && !wasBuilt) this.frame.src = this.previewUrl;
  this.request.update(site, isLive(this.agentState?.run));
  this.brain.update(site);
}

The test

Start the server with the context length you gave Ollama. With 65,536 you can leave the option out; the test used 32k:

~/sitesmith
$ npm run server -- --local-context 32768

The test site was a deliberately tiny one, "Hello Local", with a single request: Create index.html with a heading Hello from a local model and one short paragraph. Before it went off, the site was switched to "Local: qwen3:1.7b". This is what its session got:

~/sitesmith
$ tmux -L sitesmith show-environment -t =tiny-localANTHROPIC_API_KEY=ANTHROPIC_AUTH_TOKEN=ollamaANTHROPIC_BASE_URL=http://127.0.0.1:11434ANTHROPIC_DEFAULT_HAIKU_MODEL=qwen3:1.7bANTHROPIC_DEFAULT_OPUS_MODEL=qwen3:1.7bANTHROPIC_DEFAULT_SONNET_MODEL=qwen3:1.7bAPI_FORCE_IDLE_TIMEOUT=0API_TIMEOUT_MS=3600000CLAUDE_CODE_AUTO_MODE_SERVER=0CLAUDE_CODE_MAX_CONTEXT_TOKENS=32768CLAUDE_CODE_PROVIDER_MANAGED_BY_HOST=1CLAUDE_CODE_SUBAGENT_MODEL=qwen3:1.7b# … (SITESMITH_DATA, SITESMITH_SITE)

All the variables of env, the API key empty. Claude Code's banner in the terminal said qwen3:1.7b · API Usage Billing: it ran on the token, not on your plan, and nothing is billed, since the requests go to Ollama. Ollama's documentation suggests /status, typed in the terminal, to check where a session is connected.

Then the request went off. The first try ran before env had the two timeout variables, so Claude Code's defaults applied. Ollama's log:

~/sitesmith
$ docker logs ollama# …slot … new prompt, n_ctx_slot = 32768, n_keep = 4, task.n_tokens = 19602slot … prompt processing, n_tokens =  1024, progress = 0.05, t =   5.21 s / 196.66 tokens per secondslot … prompt processing, n_tokens =  4096, progress = 0.21, t =  43.48 s /  94.20 tokens per secondslot … prompt processing, n_tokens =  8192, progress = 0.42, t = 152.69 s /  53.65 tokens per secondslot … prompt processing, n_tokens = 14336, progress = 0.73, t = 505.79 s /  28.34 tokens per second[GIN] … | 500 | 10m0s | POST "/v1/messages?beta=true"

Read it from the top. The request: 19,602 tokens. Then the reading, at first almost 200 tokens a second. The figures are averages since the start, and they fall fast: 94 after 4,000 tokens, 54 after 8,000, 28 after 14,000. Every new token is weighed against all the ones before it, so the further the model gets, the slower it reads, and a CPU feels that far more than a graphics card. At 73 percent eight and a half minutes had passed. The model read on, and at ten minutes Claude Code's timeout ended the request: the 500 in the last line, at exactly 10m0s.

Claude Code showed API error · retrying… · attempt 1/10 and sent the request again. Ollama keeps what it has read in a cache, and the same request picks it up, so the retry started at 84 percent. It got to 94 percent before the next ten minutes were up, and so on. That is what API_TIMEOUT_MS is for. With an hour, in a new session, whose request matched only the beginning of what Ollama had in its cache:

~/sitesmith
$ docker logs ollama# …slot … new prompt, n_ctx_slot = 32768, n_keep = 4, task.n_tokens = 19721slot … prompt eval time = 1401705.03 ms / 14730 tokens (95.16 ms per token, 10.51 tokens per second)slot … eval time = 159215.52 ms / 576 tokens (276.90 ms per token, 3.61 tokens per second)[GIN] … | 200 | 26m10s | POST "/v1/messages?beta=true"

23 minutes for the remaining 14,730 tokens, 10.5 a second on average. Then 576 tokens of answer at 3.6 a second, two and a half minutes. Altogether 26 minutes and 10 seconds for the first answer, while Ollama's container used about 3.5 GB of memory.

The site "Hello Local" on "Local: qwen3:1.7b". In the terminal, after 26 minutes, the model promises to follow rules for commit messages and pull requests instead of building the page it was asked for. The status line at the bottom comes from the tester's own Claude Code settings: 60 percent of the 32k context used.Open the screenshot in full size
The site "Hello Local" on "Local: qwen3:1.7b". In the terminal, after 26 minutes, the model promises to follow rules for commit messages and pull requests instead of building the page it was asked for. The status line at the bottom comes from the tester's own Claude Code settings: 60 percent of the 32k context used.

And the answer was useless. The model did not read .sitesmith/request.md, called no tool and wrote no file. Instead it announced that it would follow "the guidelines in .sitesmith/request.md for attribution and compliance", and listed rules for commit messages and pull requests. Those rules are in Claude Code's own system prompt. Of the almost 20,000 tokens it had read, the model picked a passage that had nothing to do with the request, and answered that.

Sitesmith did its part. The Stop hook brought the answer into the page's feed as Claude Code's words, the session counted as hooked, and "working" went off at the end, all as with Claude. A question or a permission prompt never came up, for the simple reason that the model never tried to do anything.

What to expect, and what to try

So be clear about what you have: a way to run Claude Code on a local model, and it works. Whether it is useful depends entirely on the model and the machine, and a model of 1.7 billion parameters on a CPU is a test, not a tool. Two things went wrong here, and they are separate:

  • Too slow. Claude Code sends close to 20,000 tokens before it says anything, and more with every file it reads. Reading is what a CPU does slowly. A graphics card with enough memory for the model and its context reads much faster; how much faster depends on the card and the model, and was not measured here.

  • Too small. Claude Code's work is a long chain of decisions: read the request, look at the folder, write a file, check it, each time with the right tool and the right arguments. A model this size gets lost in 20,000 tokens of instructions, as this one did. The larger the model, the better its chances, and the more memory it needs.

What to try instead, from close to home to further away:

  • A GPU. If you have a graphics card, run Ollama where it can use it. In Docker on Linux that means passing the card through to the container; Ollama's Docker instructions show how for NVIDIA and AMD. On a Mac, run the Ollama app itself instead of Docker, because containers there do not get the GPU. Then check with ollama ps that the model sits on the GPU (its PROCESSOR column), and not partly on the CPU.

  • A bigger model with tools and 64k. Ollama's page on Claude Code names models for it (at the time of writing qwen3.5 to run locally) and links to the models with tool support. Pull one, set OLLAMA_CONTEXT_LENGTH=65536, start Sitesmith without --local-context, and pick the model in the list.

  • Ollama's cloud models. Ollama can also run models on its own servers, with an Ollama account, and its Claude Code page shows them in the same setup. A model your local Ollama lists appears in the picker like any other. This was not tried here, and with it the request and the site's files leave your machine, just as they do with Claude.

  • Your plan. For building sites, Claude is the brain this series was made for. The same picker switches back.

What you have now

A second brain per site, one choice in the header: Claude on your plan, or any model Ollama has, with Claude Code and everything Sitesmith built around it unchanged. You have seen the whole path work, and you have seen what a model too small for the job does with it, so you know what to look for when you try a bigger one.

So far Sitesmith can only be reached from the machine it runs on. In the last part it opens up to your network, behind a token, so you can look after your sites from your phone.

Comments

The comments are GitHub Discussions: sign in with GitHub to write one.

The comments come from GitHub, through giscus. Loading them sends your IP address to giscus.app and GitHub.