B2 · How Artificial Intelligence Works: Data and Learning

How Computers Hear Sound

Sound is a wave of vibrations in the air. A microphone turns the wave into an electrical signal, and the computer measures that signal thousands of times every second. Each measurement, called a sample, is stored as a number, so a recording becomes a long list of numbers.

6 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

Can a computer 'see' your voice?

Sound is a wave of vibrations in the air. A microphone turns the wave into an electrical signal, and the computer measures that signal thousands of times every second. Each measurement, called a sample, is stored as a number, so a recording becomes a long list of numbers.

In simple words

Imagine drawing the wave on graph paper and writing down its height at every tick. Loud sounds make tall waves and big numbers. High-pitched sounds make waves that are packed close together. Voice assistants learn which patterns of numbers mean which words.

Why it matters

Voice assistants, song recognition, live captions and translation apps all rely on turning sound into numbers. It is the same idea as pixels: once sound is numbers, AI can learn patterns in it.

Key terms

Sound wave
A vibration that travels through the air and reaches the ear.
Frequency
How many times a wave vibrates each second. Higher frequency sounds higher in pitch.
Amplitude
The height of a wave. Greater amplitude sounds louder.
Sample
One measurement of the wave's height at one instant.
Sampling rate
How many samples are taken each second.
Hertz (Hz)
The unit of frequency: one vibration per second.

Explained in detail

What sound is

When you pluck the string of a sitar or a guitar, it vibrates. The string pushes the air next to it, that air pushes the air beyond it, and the push travels outward as a wave. When the wave reaches your ear, your eardrum vibrates in the same pattern and your brain hears a sound. Sound needs something to travel through, which is why space is silent.

Two properties of the wave matter most. Frequency is how fast it vibrates, measured in hertz. A fast vibration sounds high, like a whistle; a slow one sounds low, like a drum. People can hear from about 20 hertz to about 20,000 hertz. Amplitude is the height of the wave, and it decides loudness: a taller wave is a louder sound.

Turning a wave into numbers

A microphone contains a thin sheet that moves with the sound wave and turns the movement into a changing electrical signal. That signal is smooth and continuous, but a computer can only store separate numbers. So it measures the height of the signal at regular instants, thousands of times a second. Each measurement is a sample. Join the samples up like dots on graph paper and you get back the shape of the wave.

The number of samples taken each second is the sampling rate. A telephone call uses about 8,000 samples a second, enough to understand speech. Music on a CD uses 44,100. There is a reason for that number: to capture a vibration correctly, you must measure it at least twice during each cycle, and the highest sound people can hear vibrates about 20,000 times a second. Sampling a little more than twice as fast captures everything the ear can detect.

RecordingSamples each secondGood enough for
Telephone callAbout 8,000Understanding speech
Voice assistantsAbout 16,000Recognising words accurately
CD music44,100Everything people can hear

Smaller files

One minute of CD-quality stereo sound is more than five million numbers. As with images, sound is usually compressed. Formats such as MP3 remove the parts of the sound that human hearing is least likely to notice, such as a quiet sound that is masked by a loud one at the same moment. The file becomes about a tenth of its size, and most listeners cannot tell the difference.

How AI listens

For an AI system, a voice is a list of numbers. To make patterns easier to find, the system often turns the sound into a spectrogram: a picture that shows which frequencies are present at each moment, with time running left to right and pitch running bottom to top. Each speech sound, each bird and each musical instrument leaves its own shape in this picture. A model trained on many labelled recordings learns those shapes. This is how a phone turns your speech into text, how an app names a song from a few seconds of it, and how researchers count bird species from recordings made in a forest.

How it works

What you hearWhat the wave looks likeThe numbers
LoudTall wavesBig values, far from zero
QuietShort wavesSmall values, near zero
High pitchWaves close togetherValues change quickly
Low pitchWaves far apartValues change slowly

Try it yourself

See a voice as a wave

Change the loudness and pitch, and watch the wave and its numbers change.

Common mistakes

  • Thinking computers store sound as sound. They store numbers measured from the wave.
  • Mixing up loudness and pitch. Loudness is how tall the wave is; pitch is how close together the waves are.
  • Thinking a few samples are enough. Clear audio needs thousands every second.

Check yourself

Question 1

1. A louder sound has…

Question 2

2. What is a sample?

Question 3

3. Waves packed close together mean…

Summary

  • Sound is a wave; a microphone turns it into a signal.
  • The computer stores thousands of samples per second as numbers.
  • AI learns which patterns of numbers mean which words or sounds.

Finished reading?