Create an AI Assistant With Local Models · Part 5 of 13 in the series
Part 5: Timers, the Weather and Tools That Answer Themselves
Add timers that ring by themselves and a weather tool that asks a free service on the internet, and make the assistant twice as fast by letting tools that act speak for themselves.
Create an AI Assistant With Local ModelsPart 5 of 13
On this page
Quest objectives
- Tell tools that act from tools that look something up
- Halve the waiting time of commands
- Build timers that speak up when they are done
- Fetch the weather from a free API and hand it to the model as text
- Add a new tool without touching the loop
At the end of the last part, switching a lamp cost two requests to the model: one in which it asked for the tool, and one in which it turned "The light in the kitchen is on." into "Let there be light: the kitchen is on now." The second request made the answer nicer and the wait twice as long.
Two kinds of tools
Look at what tools do, and there are two kinds:
Tools that act: switch a lamp, start a timer. When they are done, everything worth saying is known: "The light in the kitchen is on."
Tools that look something up: which lamps are on, what the weather will be. Their result is raw material, and the model has to read it to answer the question that was actually asked.
So let the first kind speak for itself. A tool now hands back one of two things: say with a finished sentence, or info with something for the model to read. Replace src/tools.ts:
import { lightTools } from './tools/lights.ts';
import { timerTools } from './tools/timers.ts';
import { weatherTools } from './tools/weather.ts';
/**
* What a tool hands back: `say` when it did something and has the answer ready, `info`
* when it looked something up that the model should put into words.
*/
export type ToolResult = { say: string } | { info: string };
export interface Tool {
name: string;
/** tells the model when to use the tool */
description: string;
/** the arguments, as a JSON schema */
parameters: Record<string, unknown>;
run(args: Record<string, unknown>): ToolResult | Promise<ToolResult>;
}
export const tools: Tool[] = [...lightTools, ...timerTools, ...weatherTools];
/** The tools in the form Ollama expects. */
export const toolDefinitions = tools.map((tool) => ({
type: 'function',
function: { name: tool.name, description: tool.description, parameters: tool.parameters },
}));
export async function runTool(name: string, args: Record<string, unknown>): Promise<ToolResult> {
const tool = tools.find((candidate) => candidate.name === name);
if (!tool) return { info: `There is no tool called ${name}.` };
try {
return await tool.run(args);
} catch (error) {
return { info: `The tool failed: ${(error as Error).message}` };
}
}In src/tools/lights.ts, change what the two run functions hand back:
run(args) {
// the model wrote these arguments, so check every one of them
const wanted = Array.isArray(args.rooms) ? args.rooms : [];
const rooms = wanted.filter((room) => roomNames.includes(room));
// a room that does not exist must not become "the whole home"
if (rooms.length < wanted.length) return { info: `Unknown room. The rooms are: ${roomNames.join(', ')}.` };
const change: Change = {};
if (typeof args.on === 'boolean') change.on = args.on;
if (typeof args.brightness === 'number') change.brightness = args.brightness;
if (typeof args.color === 'string' && colorNames.includes(args.color)) change.color = args.color;
const changed = setLamps(rooms, change);
return { say: say.lights(changed, change) };
},
},
{
name: 'get_lights',
description: 'Looks up which lamps are on, how bright they are and which color they have.',
parameters: { type: 'object', properties: {} },
run: () => ({ info: JSON.stringify(getLamps()) }),
},Changing the lamps is a say. The unknown room is an info: the model should read it and try again. And in src/model.ts, the loop learns the difference. Replace everything from the comment above the for to the end of the file:
// a call, its result, another call ...: after four rounds it is enough
for (let round = 0; round < 4; round += 1) {
const answer = await chat(messages, { tools: toolDefinitions });
messages.push(answer);
if (!answer.tool_calls?.length) return done(spoken(answer.content));
const said: string[] = [];
for (const call of answer.tool_calls) {
const result = await runTool(call.function.name, call.function.arguments);
if ('say' in result) said.push(result.say);
messages.push({
role: 'tool',
tool_name: call.function.name,
content: 'say' in result ? result.say : result.info,
});
}
// every tool had its answer ready: no need to ask the model again
if (said.length === answer.tool_calls.length) return done(said.join(' '));
}
return done(say.beyondMe);
}If every tool the model called had its answer ready, the turn is over and their sentences are the reply. Only when something was looked up does the loop go around again. A lamp command now costs one request where it cost two, so it takes about half as long. No faster computer gives you a speed-up like that as cheaply.
You lose the model's jokes about lamps. For an assistant you talk to twenty times a day, that is a gain as well.
Speaking without being asked
A timer answers twice: once when it is set, and once, minutes later, when it rings. For the second time nobody has asked anything. Pixel needs a way to speak up by itself. Create src/announce.ts:
/**
* Things Pixel says without being asked, such as a timer that is up. Whoever shows or speaks
* Pixel's words (the terminal, later the page) listens here.
*/
type Listener = (speech: string) => void;
const listeners: Listener[] = [];
export function onAnnounce(listener: Listener): void {
listeners.push(listener);
}
export function announce(speech: string): void {
for (const listener of listeners) listener(speech);
}Whoever shows Pixel's words registers with onAnnounce, and whoever has something to say calls announce. The timers do not need to know whether there is a terminal, a page or a loudspeaker on the other side.
Timers
Create src/timers.ts:
import { announce } from './announce.ts';
import { LOCALE, say } from './language.ts';
interface Timer {
label: string;
/** when it rings, in milliseconds since 1970 */
ends: number;
handle: NodeJS.Timeout;
}
const timers: Timer[] = [];
/** "10 minutes", "1 minute", "30 seconds", in Pixel's language. */
export function length(minutes: number): string {
const [value, unit] = minutes < 1 ? [Math.round(minutes * 60), 'second'] : [minutes, 'minute'];
return new Intl.NumberFormat(LOCALE, { style: 'unit', unit, unitDisplay: 'long', maximumFractionDigits: 1 }).format(value);
}
export function startTimer(minutes: number, label: string): void {
const timer: Timer = {
label,
ends: Date.now() + minutes * 60_000,
handle: setTimeout(() => {
timers.splice(timers.indexOf(timer), 1);
announce(say.timerDone(label));
}, minutes * 60_000),
};
timers.push(timer);
}
/** The running timers with the minutes they have left, the next one first. */
export function listTimers(): Array<{ label: string; minutesLeft: number }> {
return timers
.map((timer) => ({ label: timer.label, minutesLeft: Math.round((timer.ends - Date.now()) / 6000) / 10 }))
.sort((a, b) => a.minutesLeft - b.minutesLeft);
}
/** Stops the timer with this label, or the next one to ring when there is no such label. */
export function cancelTimer(label: string): string | null {
const timer =
timers.find((candidate) => label && candidate.label.toLowerCase() === label.toLowerCase()) ??
timers.toSorted((a, b) => a.ends - b.ends)[0];
if (!timer) return null;
clearTimeout(timer.handle);
timers.splice(timers.indexOf(timer), 1);
return timer.label;
}setTimeout does the waiting. When the time is up, the timer removes itself from the list and announces that it is done. The timers live in memory, so they are gone when the program ends. length writes a duration the way the language does, again with Intl.
The tools around it, in src/tools/timers.ts:
import { say } from '../language.ts';
import { cancelTimer, length, listTimers, startTimer } from '../timers.ts';
import type { Tool } from '../tools.ts';
export const timerTools: Tool[] = [
{
name: 'set_timer',
description: 'Starts a timer that rings after the given time.',
parameters: {
type: 'object',
properties: {
minutes: { type: 'number', description: 'how long, in minutes; 0.5 is thirty seconds' },
label: { type: 'string', description: 'what the timer is for, such as "pasta"; may be empty' },
},
required: ['minutes'],
},
run(args) {
const minutes = Number(args.minutes);
if (!(minutes > 0 && minutes <= 24 * 60)) return { info: 'minutes must be between 0 and 1440.' };
const label = typeof args.label === 'string' ? args.label : '';
startTimer(minutes, label);
return { say: say.timerSet(length(minutes), label) };
},
},
{
name: 'list_timers',
description: 'Looks up the running timers and the minutes each has left.',
parameters: { type: 'object', properties: {} },
run: () => ({ info: JSON.stringify(listTimers()) }),
},
{
name: 'cancel_timer',
description: 'Stops a running timer.',
parameters: {
type: 'object',
properties: { label: { type: 'string', description: 'the label of the timer; empty for the next one to ring' } },
},
run(args) {
const label = cancelTimer(typeof args.label === 'string' ? args.label : '');
return { say: label === null ? say.noTimer : say.timerGone(label) };
},
},
];Setting and cancelling are actions and answer with say. Listing is a lookup. The check in set_timer looks strange but is on purpose: !(minutes > 0 && ...) is also true when minutes is not a number at all, which a plain minutes <= 0 would let through.
The sentences belong in src/language.ts, together with what the weather needs in a moment:
/** The labels of the note that goes in front of every request. */
export const LABELS = {
// ...
wind: 'wind',
rain: 'chance of rain',
to: 'to',
};
/** WMO weather codes, as Open-Meteo sends them: up to this code, say this. */
export const WEATHER: Array<[number, string]> = [
[1, 'clear'],
[2, 'partly cloudy'],
[3, 'overcast'],
[48, 'fog'],
[57, 'drizzle'],
[67, 'rain'],
[77, 'snow'],
[82, 'rain showers'],
[86, 'snow showers'],
[99, 'thunderstorm'],
];
/** The sentences Pixel says without the model. */
export const say = {
// ...
timerSet: (length: string, label: string) => (label ? `Timer set: ${length} for ${label}.` : `Timer set for ${length}.`),
timerDone: (label: string) => (label ? `Time is up: ${label}!` : 'Time is up!'),
timerGone: (label: string) => (label ? `Timer for ${label} cancelled.` : 'Timer cancelled.'),
noTimer: 'No timer is running.',
};The // ... stands for what is already there: add the new lines to LABELS and to say, and WEATHER as a whole.
The weather
The model knows nothing about today's weather, but Open-Meteo does. It is a free weather service that needs no account and no key. Two requests are needed: one that turns a place name into coordinates, and one that gets the forecast for them. Create src/weather.ts:
import { LABELS, LOCALE, WEATHER } from './language.ts';
interface Place {
name: string;
country: string;
latitude: number;
longitude: number;
}
interface Forecast {
current: { temperature_2m: number; weather_code: number; wind_speed_10m: number };
daily: {
time: string[];
weather_code: number[];
temperature_2m_min: number[];
temperature_2m_max: number[];
precipitation_probability_max: number[];
};
}
const language = LOCALE.slice(0, 2);
const weekday = new Intl.DateTimeFormat(LOCALE, { weekday: 'long' });
async function getJson<T>(url: string): Promise<T> {
const response = await fetch(url, { signal: AbortSignal.timeout(5000) });
if (!response.ok) throw new Error(`${new URL(url).host} answered ${response.status}`);
return (await response.json()) as T;
}
const sky = (code: number): string => WEATHER.find(([upTo]) => code <= upTo)?.[1] ?? '';
/** The weather now and for three days, as a few lines of text the model can read. */
export async function weather(placeName: string): Promise<string> {
const found = await getJson<{ results?: Place[] }>(
`https://geocoding-api.open-meteo.com/v1/search?count=1&language=${language}&name=${encodeURIComponent(placeName)}`,
);
const place = found.results?.[0];
if (!place) return `No place called ${placeName} was found.`;
const forecast = await getJson<Forecast>(
'https://api.open-meteo.com/v1/forecast?timezone=auto&forecast_days=3' +
`&latitude=${place.latitude}&longitude=${place.longitude}` +
'¤t=temperature_2m,weather_code,wind_speed_10m' +
'&daily=weather_code,temperature_2m_min,temperature_2m_max,precipitation_probability_max',
);
const { current, daily } = forecast;
const days = daily.time.map(
(date, i) =>
`${weekday.format(new Date(`${date}T12:00`))}: ${Math.round(daily.temperature_2m_min[i]!)} ${LABELS.to} ` +
`${Math.round(daily.temperature_2m_max[i]!)} °C, ${sky(daily.weather_code[i]!)}, ` +
`${LABELS.rain} ${daily.precipitation_probability_max[i]} %`,
);
return [
`${place.name}, ${place.country}`,
`${LABELS.now}: ${Math.round(current.temperature_2m)} °C, ${sky(current.weather_code)}, ` +
`${LABELS.wind} ${Math.round(current.wind_speed_10m)} km/h`,
...days,
].join('\n');
}The function does not hand the model the service's raw JSON, which is long and full of numbers with units somewhere else. It writes a few plain lines:
Vienna, Austria
Now: 7 °C, overcast, wind 10 km/h
Tuesday: 5 to 14 °C, rain, chance of rain 100 %
Wednesday: 10 to 15 °C, rain showers, chance of rain 55 %
Thursday: 7 to 12 °C, rain, chance of rain 15 %This is worth remembering: write tool results the way you would write a note for a colleague. A small model reads a short text far more reliably than nested data, and every token it does not have to read makes the answer faster. The weather codes are turned into words for the same reason, and each day gets its weekday, because you already know that the model should not have to count days.
AbortSignal.timeout(5000) gives up after five seconds, so a slow service cannot hang the assistant. Add your home town to src/config.ts:
/** The place the weather is for when nobody names one. */
export const HOME_PLACE = process.env.HOME_PLACE ?? 'Vienna';And the tool, in src/tools/weather.ts:
import { HOME_PLACE } from '../config.ts';
import type { Tool } from '../tools.ts';
import { weather } from '../weather.ts';
export const weatherTools: Tool[] = [
{
name: 'get_weather',
description: 'Looks up the weather now and the forecast for today and the next two days.',
parameters: {
type: 'object',
properties: {
place: { type: 'string', description: `a town or city; leave it out for home (${HOME_PLACE})` },
},
},
async run(args) {
const place = typeof args.place === 'string' && args.place.trim() ? args.place : HOME_PLACE;
return { info: await weather(place) };
},
},
];Hearing the timer
The new tools are already in the list in src/tools.ts, and the loop needs no change at all: a new tool is a description and a function. Only the chat has to listen for announcements. In src/chat.ts, add the import import { onAnnounce } from './announce.ts'; and these lines above the greeting:
// a timer that is up, for example
onAnnounce((speech) => console.log(`\n${name}> ${speech}`));$ npm run chatTalk to Pixel. /bye ends the chat.you> Make the bedroom cosy.pixel> The light in the bedroom is warm white at 30 percent. living room ○ kitchen ○ bedroom ● 30% warm white bathroom ○ hallway ○you> Set a timer for the tea, half a minute.pixel> Timer set: 30 seconds for tea.you> Do I need an umbrella today?pixel> Yes, take one: it's overcast at 7 degrees now, and rain is almost certain today.you>pixel> Time is up: tea!
The lamp and the timer were answered by their tools, in one request each. The umbrella needed two: the model called get_weather, read the lines and answered the question that was asked, which was not "what is the weather" but "do I need an umbrella". And half a minute later Pixel spoke up by itself.
Pixel now has six tools and one request per command. But even one request to a language model takes a second or more, and "kitchen light on" really needs no thinking at all. In the next part, simple commands stop bothering the model.
Comments
The comments are GitHub Discussions: sign in with GitHub to write one.
The comments come from GitHub, through giscus. Loading them sends your IP address to giscus.app and GitHub.