Create an AI Assistant With Local Models · Part 3 of 13 in the series
Part 3: A Persona and a Prompt That Stays Fast
Turn the model into Pixel with a system prompt, tell it the date and time with every request, keep the conversation short, and arrange the prompt so that the model can reuse its work.
Create an AI Assistant With Local ModelsPart 3 of 13
On this page
Quest objectives
- Give the model a persona with a system prompt
- Keep everything that depends on the language in one file
- Hand the model the date, the time and a small calendar
- Keep only the last few exchanges
- Order the prompt so that Ollama can reuse what it has already read
The chat from the last part talks like every chatbot: friendly, long-winded, and about anything. An assistant that will one day speak its answers in your kitchen needs to be short, and it should have a character of its own. Both come from the same place: text that your program puts in front of the conversation.
The system prompt
A message with the role system comes first in the list and tells the model who it is and how it should behave. The model treats it as the rules of the conversation. This is Pixel's. Create src/language.ts:
/**
* Everything that depends on the language Pixel speaks: its persona, the words it listens
* for and the sentences it says itself. To teach Pixel another language, translate this file.
*/
export const LOCALE = 'en-US';
export const PERSONA = `
You are Pixel, the assistant of a small home. You talk with the people who live here.
How you talk:
- Short and friendly, with a dry sense of humor. One or two sentences.
- Plain spoken language: no lists, no markdown, no emojis.
- You answer in English.
What you do:
- If you don't know something, say so in one sentence. Don't guess.
`.trim();
/** The labels of the note that goes in front of every request. */
export const LABELS = {
now: 'Now',
nextDays: 'The next days',
};A few things make a system prompt work, especially with a small model:
Say what you want, plainly. "One or two sentences" works. "Be concise" does not.
Keep it short. Every line is read again with every request, and a small model starts to ignore rules when there are twenty of them.
Give it a way out. The last line allows the model to say that it does not know. Without it, a model prefers a confident invention.
The file is called language.ts because it will hold everything that depends on the language Pixel speaks: its persona now, and later the words it listens for and the sentences it says by itself. Teaching Pixel another language then means translating this one file, and little else.
Add the assistant's name to src/config.ts:
/** The settings of the assistant, in one place. An environment variable wins over the default. */
export const NAME = 'Pixel';
export const OLLAMA_URL = process.env.OLLAMA_URL ?? 'http://localhost:11434';
export const MODEL = process.env.MODEL ?? 'qwen3:4b-instruct';What the model cannot know
Ask the chat from the last part what day it is, and it will apologize or invent one. The model is a file from the past. Whatever is true now has to be in the request: the date, the time, later the weather and what Pixel remembers about you. Create src/prompt.ts:
import { LABELS, LOCALE } from './language.ts';
const day = new Intl.DateTimeFormat(LOCALE, { weekday: 'long', year: 'numeric', month: 'long', day: 'numeric' });
const time = new Intl.DateTimeFormat(LOCALE, { hour: '2-digit', minute: '2-digit', hour12: false });
/**
* What changes from one request to the next: the date and time and a small calendar. It
* goes in front of the user's words, not into the system prompt.
*/
export function contextBlock(now = new Date()): string {
const lines = [`${LABELS.now}: ${day.format(now)}, ${time.format(now)}`];
// small models are bad at counting weekdays, so the next days come ready-made
const next = Array.from({ length: 6 }, (_, i) => day.format(new Date(now.getTime() + (i + 1) * 86_400_000)));
lines.push(`${LABELS.nextDays}: ${next.join('; ')}`);
return `[${lines.join('\n')}]`;
}
/** Makes an answer fit to be spoken: no markdown, links or emojis, and at most two sentences. */
export function spoken(text: string, maxSentences = 2): string {
const plain = text
.replace(/```[\s\S]*?```/g, ' ')
.replace(/https?:\/\/\S+/g, '')
.replace(/[*_#`>|]/g, '')
.replace(/\p{Extended_Pictographic}/gu, '')
.replace(/\s+/g, ' ')
.trim();
const sentences = plain.match(/[^.!?]+[.!?]+(?=\s|$)|[^.!?]+$/g) ?? [];
return sentences.slice(0, maxSentences).map((sentence) => sentence.trim()).join(' ');
}contextBlock writes a small note that goes in front of what the user said. The model then sees this as the user's message:
[Now: Tuesday, November 17, 2026, 09:14
The next days: Wednesday, November 18, 2026; Thursday, November 19, 2026; Friday, November 20, 2026; Saturday, November 21, 2026; Sunday, November 22, 2026; Monday, November 23, 2026]
What is the date on Friday?The second line looks odd, but it matters. Small models cannot do calendar arithmetic: asked for the date on Friday, they guess. With the next days written out, the model only has to find Friday, and that it can do. This is a pattern you will meet again and again: do not ask the model to work out what a line of code can work out for it.
Intl.DateTimeFormat is built into JavaScript and writes dates the way a language does, which is why LOCALE is part of language.ts.
spoken is the safety net behind the system prompt. A model that was told "no markdown, two sentences" still slips now and then, so the function removes what cannot be spoken and cuts the answer after two sentences.
A short memory
The chat from the last part sent the whole conversation every time. That gets slower with every exchange, because the model reads all of it again, and one day it no longer fits into the context window. For an assistant the last few exchanges are enough: they let it understand "and in the kitchen too". Create src/history.ts:
import type { Message } from './ollama.ts';
/** The last few exchanges, so the model understands "and the kitchen too". */
const turns: Message[] = [];
export function recent(): Message[] {
return turns;
}
export function keep(text: string, speech: string): void {
turns.push({ role: 'user', content: text }, { role: 'assistant', content: speech });
// three exchanges are enough, and a short prompt is a fast prompt
turns.splice(0, Math.max(0, turns.length - 6));
}Asking the model
Now put it together in src/model.ts:
import { keep, recent } from './history.ts';
import { PERSONA } from './language.ts';
import { chat, type Message } from './ollama.ts';
import { contextBlock, spoken } from './prompt.ts';
export interface Reply {
speech: string;
}
/** Asks the local model and returns its answer, ready to be spoken. */
export async function askModel(text: string): Promise<Reply> {
const messages: Message[] = [
// first what never changes, so Ollama can reuse its work from the last request ...
{ role: 'system', content: PERSONA },
...recent(),
// ... and last what is new
{ role: 'user', content: `${contextBlock()}\n${text}` },
];
const answer = await chat(messages);
const speech = spoken(answer.content);
keep(text, speech);
return { speech };
}Look at the order of the messages. It is not an accident.
Before a model can write, it has to read the whole prompt, token by token, and that takes time. Ollama remembers what it read for the last request, and if the next request starts the same way, it skips that part. So the rule is: what never changes goes first, what changes goes last. The system prompt is the same every time, so it leads. The date and time change every minute, so they go at the very end, in front of the user's words, and not into the system prompt, where they would spoil the start of every request. On the real assistant, reading a turn of about 1,700 tokens takes around 150 milliseconds this way.
Notice also that keep stores the user's words without the note. The note is only true for one moment.
The answers are short now, so they no longer need to appear piece by piece, and from here on the assistant waits for the whole answer. There is a better reason too, which you will meet in part 7: Pixel is going to check an answer before anyone sees it, and you cannot take back half a sentence. In src/ollama.ts, give chat two options:
/** Sends the conversation to the model and returns its whole answer. */
export async function chat(messages: Message[]): Promise<Message> {
const response = await post('/api/chat', {
model: MODEL,
messages,
stream: false,
think: false,
options: { temperature: 0.2, num_predict: 300 },
});
const data = (await response.json()) as { message: Message };
return data.message;
}temperature is how much chance the model uses when it picks the next token. At 0.2 it mostly takes the likeliest one, which makes an assistant more reliable; near 1 it gets more inventive and more wrong. num_predict stops the model after 300 tokens, so an answer that runs away cannot keep you waiting for a minute.
The chat, with Pixel in it
Replace src/chat.ts:
import { createInterface } from 'node:readline/promises';
import { NAME } from './config.ts';
import { askModel } from './model.ts';
const terminal = createInterface({ input: process.stdin, output: process.stdout });
// Ctrl+D closes the input: leave without an error
terminal.on('close', () => process.exit());
const name = NAME.toLowerCase();
console.log(`Talk to ${NAME}. /bye ends the chat.\n`);
while (true) {
const text = (await terminal.question('you> ')).trim();
if (text === '/bye') break;
if (!text) continue;
const answer = await askModel(text);
console.log(`${name}> ${answer.speech}`);
}
terminal.close();$ npm run chatTalk to Pixel. /bye ends the chat.you> Who are you?pixel> I'm Pixel, the assistant of this home. Small, but I try.you> What is the date on Friday?pixel> Friday is November 20.you> Write me a poem about lamps with ten verses.pixel> Lamps glow so we don't walk into furniture. That's the short version, and the only one you'll get from me.you> /bye
Same model, same machine, a different character. It knows the date, it keeps it short, and it has opinions about poems.
Pixel can talk now, but it cannot do anything. Ask it to switch on a lamp and it will happily say it did. In the next part you give it a home with lamps and the most important skill of an assistant: calling tools.
Comments
The comments are GitHub Discussions: sign in with GitHub to write one.
The comments come from GitHub, through giscus. Loading them sends your IP address to giscus.app and GitHub.