Create an AI Assistant With Local Models · Part 2 of 13 in the series
Part 2: Talking to the Model From Code
Send your first request to the local model from TypeScript, learn what a conversation looks like to a model, and build a small chat in the terminal that shows the answer while it is written.
Create an AI Assistant With Local ModelsPart 2 of 13
On this page
Quest objectives
- Set up a TypeScript project that Node.js runs without a build step
- Send a request to Ollama and read the answer
- Understand messages, roles and why the model has no memory
- Show an answer piece by piece while the model writes it
- Build a chat loop in the terminal
In the first part you talked to the model in Ollama's own chat. Now you write the program on the other end yourself. It is less than you may think: the model sits behind a web address on your computer, and a conversation is a list of messages that you send there.
The project
You need Node.js, version 24 or newer. Check yours:
$ node --versionv26.13.0
Create a folder for the assistant and install TypeScript:
$ mkdir pixel-assistant$ cd pixel-assistant$ mkdir src$ npm install --save-dev typescript @types/nodeadded 4 packages, and audited 5 packages in 1s
The series uses TypeScript, which is JavaScript with types: you say what shape your data has, and the editor tells you about mistakes before the program runs. Node.js runs .ts files directly, by ignoring the types, so there is nothing to compile. Create package.json in the folder (if npm already made one, replace everything but the devDependencies):
{
"name": "pixel-assistant",
"private": true,
"type": "module",
"scripts": {
"chat": "node src/chat.ts",
"typecheck": "tsc"
}
}"type": "module" lets the files use import and export. The two scripts are shortcuts you will use in a moment. The type checker needs a few settings of its own, in tsconfig.json:
{
"compilerOptions": {
"module": "nodenext",
"target": "esnext",
"types": ["node"],
"strict": true,
"noEmit": true,
"allowImportingTsExtensions": true,
"erasableSyntaxOnly": true,
"verbatimModuleSyntax": true,
"skipLibCheck": true
},
"include": ["src"]
}You do not need to know these by heart. They tell TypeScript to only check the code and write nothing (noEmit), and to insist on the kind of TypeScript that Node.js can run as it is.
The first request
Create src/hello.ts:
const response = await fetch('http://localhost:11434/api/chat', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({
model: 'qwen3:4b-instruct',
messages: [{ role: 'user', content: 'Say hello in five words.' }],
stream: false,
}),
});
const data = await response.json();
console.log(data.message.content);$ node src/hello.tsHello there, nice to meet you!
That is a complete program that talks to a language model. fetch sends a request to Ollama's address, as a browser would to a website. The request names the model and carries the conversation so far, here a single message. stream: false asks for the whole answer in one piece. What comes back is JSON; the program printed one field of it. This is all of it:
{
"model": "qwen3:4b-instruct",
"created_at": "2026-11-17T08:12:45.416799Z",
"message": {
"role": "assistant",
"content": "Hello there, nice to meet you!"
},
"done": true,
"done_reason": "stop",
"total_duration": 1191566416,
"load_duration": 2154458,
"prompt_eval_count": 16,
"prompt_eval_duration": 83809000,
"eval_count": 9,
"eval_duration": 399921000
}The answer is a message like the one you sent, with the role assistant. The numbers below it tell you what happened: the model read 16 tokens (prompt_eval_count) and wrote 9 (eval_count). The durations are in nanoseconds, so writing took 0.4 seconds here. Divide the tokens written by that time and you have the model's speed on your machine in tokens per second, the number to watch when you try another model.
Messages and roles
Every message has a role that says who is speaking:
user: the person talking to the model.assistant: the model's own earlier answers.system: instructions from the program around the model. You will use it in the next part to make the model into Pixel.
Remember from the first part: the model keeps nothing between two requests. If it should know what was said a minute ago, you have to send it again, every time, as the list of messages. A chat is a program that keeps that list and sends it whole with each new question.
A small client
The address and the model's name should not be scattered through the code. Put them in src/config.ts:
/** The settings of the assistant, in one place. An environment variable wins over the default. */
export const OLLAMA_URL = process.env.OLLAMA_URL ?? 'http://localhost:11434';
export const MODEL = process.env.MODEL ?? 'qwen3:4b-instruct';process.env holds the environment variables, so you can try another model later with MODEL=llama3.2:3b npm run chat without touching the code. Now the part that talks to Ollama, src/ollama.ts:
import { MODEL, OLLAMA_URL } from './config.ts';
export interface Message {
role: 'system' | 'user' | 'assistant';
content: string;
}
async function post(path: string, body: object): Promise<Response> {
const response = await fetch(`${OLLAMA_URL}${path}`, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(body),
});
if (!response.ok) throw new Error(`Ollama answered ${response.status}: ${await response.text()}`);
return response;
}
/** Sends the conversation to the model and returns its whole answer. */
export async function chat(messages: Message[]): Promise<Message> {
const response = await post('/api/chat', { model: MODEL, messages, stream: false, think: false });
const data = (await response.json()) as { message: Message };
return data.message;
}
/** Sends the conversation to the model and hands out its answer piece by piece. */
export async function* chatStream(messages: Message[]): AsyncGenerator<string> {
const response = await post('/api/chat', { model: MODEL, messages, stream: true, think: false });
const decoder = new TextDecoder();
let rest = '';
for await (const chunk of response.body!) {
rest += decoder.decode(chunk, { stream: true });
// one JSON object per line; the last line may still be incomplete
const lines = rest.split('\n');
rest = lines.pop()!;
for (const line of lines) {
if (!line.trim()) continue;
const part = JSON.parse(line) as { message: Message };
yield part.message.content;
}
}
}From the top:
Messagedescribes one message. From now on the editor complains if you misspell a role.postsends a request and turns an error answer from Ollama, such as a model that is not downloaded, into an error you can read.chatis whathello.tsdid: send the messages, get the whole answer.chatStreamasks for the answer piece by piece, the way you saw it appear in the first part.
How streaming works
With stream: true, Ollama does not wait until the answer is done. It sends one line of JSON for every piece, as soon as the model has written it, and a last line with "done": true and the numbers:
{"model":"qwen3:4b-instruct","created_at":"...","message":{"role":"assistant","content":"Hello"},"done":false}
{"model":"qwen3:4b-instruct","created_at":"...","message":{"role":"assistant","content":" there"},"done":false}
{"model":"qwen3:4b-instruct","created_at":"...","message":{"role":"assistant","content":","},"done":false}
...
{"model":"qwen3:4b-instruct","created_at":"...","message":{"role":"assistant","content":""},"done":true,"done_reason":"stop","eval_count":9,"eval_duration":399921000}chatStream reads what arrives, cuts it at the line breaks, and hands out the content of each line. A line can arrive in two halves, so the function keeps the unfinished rest for the next round. The function* and yield make it a generator: a function that hands out many values, one after the other, which the caller collects with for await.
The chat
Create src/chat.ts:
import { createInterface } from 'node:readline/promises';
import { chatStream, type Message } from './ollama.ts';
const terminal = createInterface({ input: process.stdin, output: process.stdout });
// Ctrl+D closes the input: leave without an error
terminal.on('close', () => process.exit());
/** The whole conversation so far. The model sees all of it with every request. */
const messages: Message[] = [];
console.log('Talk to the model. /bye ends the chat.\n');
while (true) {
const text = (await terminal.question('you> ')).trim();
if (text === '/bye') break;
if (!text) continue;
messages.push({ role: 'user', content: text });
process.stdout.write('model> ');
let answer = '';
for await (const piece of chatStream(messages)) {
process.stdout.write(piece);
answer += piece;
}
process.stdout.write('\n');
messages.push({ role: 'assistant', content: answer });
}
terminal.close();The loop asks you for a line, adds it to messages, sends the whole list and prints the pieces as they come. Then it adds the model's answer to the list too, so the next request carries it along. Try the model's memory:
$ npm run chatTalk to the model. /bye ends the chat.you> My name is Robin. Remember that.model> Nice to meet you, Robin! I'll keep that in mind for the rest of our chat.you> What is my name?model> Your name is Robin.you> /bye
It knows your name because the second request contained the whole conversation. Send only the new question instead, by changing chatStream(messages) to chatStream([{ role: 'user', content: text }]), and it will have forgotten you at once.
One last command. It runs the type checker over your code, and printing nothing means it found nothing:
$ npm run typecheckYou now have a chat with a language model in fifty lines. It is polite, it is wordy, and it has no idea who it is. In the next part it becomes Pixel: it gets a persona, learns today's date, and you arrange the prompt so that the answers stay fast.
Comments
The comments are GitHub Discussions: sign in with GitHub to write one.
The comments come from GitHub, through giscus. Loading them sends your IP address to giscus.app and GitHub.