Building on Voxel Terrain in Babylon.js · Part 10 of 10 in the series
Part 10: Dust and Sound
Give the collapse weight: puffs of dust in the color of the material where pieces land and blocks break, and thuds, cracks and knocks made with the Web Audio API from noise and a falling tone, louder and deeper for heavier, harder hits, with no sound files.
Quest objectives
- Find a falling piece's hardest impact, and where it was
- Puff dust with Babylon.js particles, from a texture painted in code
- Make thuds, cracks and knocks from noise, filters and envelopes with the Web Audio API
- Make a sound softer with distance and place it left or right
- Start audio on the first click, as browsers require
The pieces fall, tumble and smash things, and they do it without a sound, without a speck of dust: like a video with the sound off. This last part adds what makes a collapse feel heavy. Where a piece lands, a puff of dust rises, bigger for a harder hit and a bigger piece. A block that breaks gives a crack and a puff in its own color. Landings thud, deeper for heavier pieces. Placing and chiseling blocks gets a dry knock.
As with the textures, nothing is loaded: the dust's texture is painted in code, and every sound is made from noise and a tone, shaped by a filter and an envelope.
Impacts
The body works out how fast each contact point closes in on its surface. A point closing in faster than 2 m/s is an impact. The body keeps the hardest one since it was last asked, and where it was. In src/body.ts, add the speed to the constants:
/** A point closing in on a surface faster than this, in meters per second, is an impact. */
const IMPACT_SPEED = 2;A field to Body:
/** The hardest impact since someone last read and cleared it: where, and how fast. */
impact: { point: Vector3; speed: number } | null = null;And in contacts(), after the hits:
if (closing > Math.max(this.impact?.speed ?? 0, IMPACT_SPEED)) this.impact = { point, speed: closing };A landing gives a few impacts, one per bounce, and a piece reports at most one per frame. In src/pieces.ts, add to the fields of Pieces:
/** Called when a falling piece hits something hard: where, how fast, and the piece. */
onImpact: (point: Vector3, speed: number, piece: Piece) => void = () => {};And in update(), after the hits are passed on:
if (piece.body.impact) this.onImpact(piece.body.impact.point, piece.body.impact.speed, piece);
piece.body.impact = null;The tools tell what they did, too. In src/buildTool.ts, add to the fields of BlockTool:
/** Called with every block the tool places or breaks. */
onEdit: (action: 'place' | 'break', block: Block) => void = () => {};And call it where the tool breaks or places a block, at the end of update():
if (action === 'break' && aim.kind === 'block') {
this.store.remove(aim.block);
this.onEdit('break', aim.block);
}
if (action === 'place' && fits) {
this.store.add(block);
this.onEdit('place', block);
}The chisel says when it changed a block: the store's revision counts every change, and a cut that changes nothing, such as rounding a round edge, leaves it as it was. In src/chiselTool.ts, add to its fields:
/** Called with every block the chisel changes. */
onCut: (block: Block) => void = () => {};And in update(), around the two lines that reshape the block, import the type Block:
const before = this.store.revision;
if (this.clicked === 'cut' && possible) this.store.reshape(block, cut.cuts, cut.slope);
if (this.clicked === 'sharpen') this.store.reshape(block, block.cuts & ~mask, block.slope);
if (this.store.revision !== before) this.onCut(block);Dust
Babylon.js has a ParticleSystem: a cloud of small billboards, each living a while, moving and fading. A puff of dust is one, made for the occasion: it sends all of its bits at once, from a ball around the point, up and out at first, and then gravity pulls them down slowly. It disposes of itself when they are gone.
The bits need a texture: a soft round spot, white, fading to see-through toward its rim, painted once into a 32 × 32 texture. Its color comes from the particle system, the material's color mixed half with grey: dust is paler than the thing it came from. The bigger the puff, the more bits, the larger and the faster. Create src/dust.ts:
import {
Color3,
Color4,
Constants,
ParticleSystem,
RawTexture,
Texture,
Vector3,
type Scene,
} from '@babylonjs/core/pure';
/** Pixels along each side of the dust's texture. */
const SIZE = 32;
/** A soft round spot, painted once: white, and see-through toward its rim. */
function paintSpot(scene: Scene): RawTexture {
const data = new Uint8Array(SIZE * SIZE * 4);
for (let y = 0; y < SIZE; y += 1) {
for (let x = 0; x < SIZE; x += 1) {
const d = Math.hypot(x + 0.5 - SIZE / 2, y + 0.5 - SIZE / 2) / (SIZE / 2);
const alpha = Math.max(1 - d, 0) ** 1.5;
data.set([255, 255, 255, alpha * 255], (y * SIZE + x) * 4);
}
}
return new RawTexture(data, SIZE, SIZE, Constants.TEXTUREFORMAT_RGBA, scene, false, false, Texture.BILINEAR_SAMPLINGMODE);
}
/** Puffs of dust, each a little particle system that goes away when it is done. */
export class Dust {
private readonly texture: RawTexture;
private readonly scene: Scene;
constructor(scene: Scene) {
this.scene = scene;
this.texture = paintSpot(scene);
}
/**
* A puff at a point, in a material's color mixed with grey. `amount` from 0 to 1 says
* how big: more, larger and faster bits for a harder hit.
*/
puff(point: Vector3, color: string, amount: number): void {
const strength = Math.min(Math.max(amount, 0), 1);
const count = Math.round(8 + 56 * strength);
const system = new ParticleSystem('dust', count, this.scene);
system.particleTexture = this.texture;
system.emitter = point.clone();
system.createSphereEmitter(0.3 + strength * 0.7);
const tint = Color3.Lerp(Color3.FromHexString(color), new Color3(0.75, 0.72, 0.7), 0.5);
system.color1 = new Color4(tint.r, tint.g, tint.b, 0.8);
system.color2 = new Color4(tint.r * 0.8, tint.g * 0.8, tint.b * 0.8, 0.65);
system.colorDead = new Color4(tint.r, tint.g, tint.b, 0);
system.minSize = 0.2 + strength * 0.3;
system.maxSize = 0.5 + strength * 0.9;
system.minLifeTime = 0.6;
system.maxLifeTime = 1.4 + strength;
system.minEmitPower = 0.3;
system.maxEmitPower = 1 + strength * 3;
system.minAngularSpeed = -1;
system.maxAngularSpeed = 1;
// Up and out at first, then the bits sink slowly.
system.gravity = new Vector3(0, -1.5, 0);
system.blendMode = ParticleSystem.BLENDMODE_STANDARD;
system.manualEmitCount = count;
system.targetStopDuration = 3;
system.disposeOnStop = true;
system.start();
}
}Particles need to be registered, like the rest of Babylon.js in this project: the system itself, and the scene component that draws it. In src/main.ts, add to the registrations, and import RegisterParticleSystem and RegisterParticleSystemComponent:
// For the dust.
RegisterParticleSystem();
RegisterParticleSystemComponent();Sound from noise
The Web Audio API builds sound from nodes connected in a graph: a source, filters, a gain that sets the loudness, and the speakers at the end. The sounds here all start from the same second of white noise, a random number per sample, made once. Noise contains every pitch at once, and a filter picks a part of it:
A thud is noise through a low-pass filter, so only the deep part comes through, with a tone underneath that falls by an octave as it fades: the body of a heavy thing landing. The heavier the piece, the deeper the tone.
A crack is noise through a high-pass filter, the bright hiss of splintering, with some of the middle for the body of the break. It is over in a quarter of a second.
A knock is a short burst of noise through a band-pass filter: a dry click. Higher for the chisel than for a block put down.
What makes noise sound like a hit is the envelope: the gain starts at full loudness at once and falls off along an exponential ramp. The ear hears the sharp start as the hit, and the fall as the material ringing out.
Each sound gets softer with distance, halved at 8 m, and is placed between the ears by a stereo panner: how far the sound lies to the camera's right decides how much of it the right speaker gets. A collapse can make dozens of sounds at once, which only turns to mush; at most six start in any tenth of a second.
Browsers let a page play sound only after the user did something on it. So the audio context is made, or resumed, on the first click: the one that captures the mouse. Create src/sounds.ts:
import { Vector3, type Camera } from '@babylonjs/core/pure';
/** How many sounds may start in one tenth of a second: a collapse makes a lot of them. */
const MOST_AT_ONCE = 6;
/**
* Thuds, cracks and knocks, made with the Web Audio API from noise and a falling tone,
* no sound files. Each goes through a filter that gives it its color and an envelope that
* makes it start at once and fade out.
*/
export class Sounds {
private context: AudioContext | null = null;
private noise: AudioBuffer | null = null;
private recent: number[] = [];
private readonly camera: Camera;
constructor(camera: Camera) {
this.camera = camera;
}
/** Browsers let a page play sound only after a click: called on the first one. */
start(): void {
if (!this.context) {
this.context = new AudioContext();
// A second of white noise, which every sound plays a piece of.
this.noise = this.context.createBuffer(1, this.context.sampleRate, this.context.sampleRate);
const samples = this.noise.getChannelData(0);
for (let i = 0; i < samples.length; i += 1) samples[i] = Math.random() * 2 - 1;
}
void this.context.resume();
}
/** Something heavy landing: deep noise and a tone falling from `pitch`, `strength` 0 to 1. */
thud(at: Vector3, strength: number, pitch: number): void {
this.play(at, strength, 0.6, (context, out, time) => {
this.noiseThrough(context, out, time, 'lowpass', pitch * 3, 0.5);
const tone = context.createOscillator();
tone.frequency.setValueAtTime(pitch, time);
tone.frequency.exponentialRampToValueAtTime(pitch * 0.5, time + 0.3);
tone.connect(out);
tone.start(time);
tone.stop(time + 0.6);
});
}
/** A block breaking: a short, bright crackle. */
crack(at: Vector3, strength: number): void {
this.play(at, strength, 0.25, (context, out, time) => {
this.noiseThrough(context, out, time, 'highpass', 1800, 0.25);
this.noiseThrough(context, out, time, 'bandpass', 600, 0.15);
});
}
/** A block put down or chiseled: a dry knock. */
knock(at: Vector3, pitch = 700): void {
this.play(at, 0.4, 0.08, (context, out, time) => this.noiseThrough(context, out, time, 'bandpass', pitch, 0.08));
}
/** Plays a piece of the noise through a filter into `out`. */
private noiseThrough(
context: AudioContext,
out: AudioNode,
time: number,
type: BiquadFilterType,
frequency: number,
length: number,
): void {
const source = context.createBufferSource();
source.buffer = this.noise;
const filter = context.createBiquadFilter();
filter.type = type;
filter.frequency.value = frequency;
source.connect(filter).connect(out);
source.start(time, Math.random() * 0.5, length);
}
/**
* Starts a sound at a point: softer the further away it is, more in the left or right
* ear the more it is to that side of the camera. `make` connects its parts to `out`,
* whose loudness falls off over `length` seconds.
*/
private play(
at: Vector3,
strength: number,
length: number,
make: (context: AudioContext, out: AudioNode, time: number) => void,
): void {
const context = this.context;
if (!context || context.state !== 'running') return;
const now = context.currentTime;
this.recent = this.recent.filter((time) => now - time < 0.1);
if (this.recent.length >= MOST_AT_ONCE) return;
this.recent.push(now);
const offset = at.subtract(this.camera.globalPosition);
const distance = offset.length();
const volume = Math.min(Math.max(strength, 0), 1) / (1 + distance / 8);
if (volume < 0.01) return;
const right = this.camera.getDirection(Vector3.Right());
const panner = context.createStereoPanner();
panner.pan.value = distance > 0 ? Vector3.Dot(offset, right) / distance : 0;
const gain = context.createGain();
gain.gain.setValueAtTime(volume, now);
gain.gain.exponentialRampToValueAtTime(0.001, now + length);
gain.connect(panner).connect(context.destination);
make(context, gain, now);
}
}Each sound plays a piece of the noise from a random point in it, so that no two sound quite the same. The tone and the noise sources stop by themselves; once stopped, the browser drops the nodes.
Wiring it up
In src/main.ts, make the dust and the sounds after the pieces, with a helper for a block's middle and one for a block that breaks. A landing puffs and thuds:
// Dust and sound where pieces land, blocks break and blocks go down.
const dust = new Dust(scene);
const sounds = new Sounds(camera);
/** The middle of a block of the grid, in the world. */
const middleOf = ({ x, y, z, size }: Block) => new Vector3(x + size / 2, y + size / 2, z + size / 2).scaleInPlace(CELL);
/** A block breaks: a crack and a puff in its color. */
const crumble = (point: Vector3, block: Block, strength: number) => {
dust.puff(point, MATERIALS[block.material].color, strength);
sounds.crack(point, strength);
};
// A landing: the harder and the bigger the piece, the more dust, and the deeper the thud.
pieces.onImpact = (point, speed, piece) => {
const strength = Math.min((speed - 2) / 8, 1);
const tonnes = piece.body.mass / 1000;
dust.puff(point, MATERIALS[piece.blocks[0].material].color, strength * Math.min(0.3 + tonnes / 10, 1));
sounds.thud(point, strength, Math.min(Math.max(110 / Math.cbrt(tonnes), 40), 140));
};A block that a falling piece breaks crumbles:
pieces.onHit = (block, speed, mass) => {
if (!damage.hit(block, speed, mass)) return;
blocks.store.remove(block);
crumble(middleOf(block), block, 0.8);
broken += 1;
};So do blocks broken off rubble and blocks you break, and placing and chiseling knock. Replace the rubble's handler after the tools are made:
blockTool.breakRubble = (piece, block) => {
crumble(piece.body.toWorld(middleOf(block)), block, 0.4);
pieces.breakBlock(piece, block);
};
blockTool.onEdit = (action, block) => {
if (action === 'place') sounds.knock(middleOf(block));
else crumble(middleOf(block), block, 0.4);
};
chiselTool.onCut = (block) => sounds.knock(middleOf(block), 1400);And start the sound on the first click:
canvas.addEventListener('pointerdown', (event) => {
sounds.start();Import Dust from ./dust.ts, Sounds from ./sounds.ts, and the type Block from ./blocks.ts.
Try it
Break the beam next to the pillar: the copper ball lands on the table with a thud, the plank top cracks, and a puff of copper-tinted dust rises where it hit. Knock out the pillar's foot and listen to it come down. Build a house, with a roof, and take it apart from below.
Where to go from here
That's the series: blocks of three sizes on the smooth terrain, painted materials lit by the terrain's cave light, walking on blocks, a chisel that rounds and slopes, the rules of what holds a block, rigid bodies that fall and land as rubble, chain reactions, and dust and sound. A few ideas for what's next:
Saving: the blocks are a list of small records, and the edits of the terrain a list of balls. Keep both in IndexedDB, as the terrain series suggested for the edits, and rubble too, with its pose.
Strength with weight: the support of part 6 is all or nothing. A beam could count the weight it carries and break when it carries too much, so that a long, thin bridge sags and gives way under a heavy load.
Pieces that hit each other: falling pieces meet the terrain, the blocks and rubble, but not each other. Testing each falling piece's points against the others, and giving the push to both, is a step toward a full physics engine.
Sleeping islands: a big collapse makes many pieces. Pieces that touch each other could be solved together, which keeps stacks of rubble from jittering, and the quiet ones could sleep until something touches them.
Many players: everything here is decided on one machine. With a server deciding what falls, and every client playing the same falls back, the same collapse happens for everyone.
The code of every part is in the demos, which play right here in the series. Have fun building, and taking it apart again.