I6 · Neural Networks and Deep Learning Essentials

Biological and Artificial Neurons

A biological neuron receives signals through dendrites, combines them in the cell body and, if the combined signal is strong enough, sends an electrical pulse along its axon to other neurons through synapses whose strengths change with learning. An artificial neuron is a mathematical simplification: it multiplies each input by a weight, adds them with a bias and passes the total through an activation function to produce its output.

10 min Intermediate · Sublevel Intermediate Jump to the activity

What you need: The earlier topics of this module.

Definition

Your brain has about 86 billion neurons, each connected to thousands of others, running on about 20 watts. An artificial neuron is a tiny caricature of one. How close is the resemblance?

A biological neuron receives signals through dendrites, combines them in the cell body and, if the combined signal is strong enough, sends an electrical pulse along its axon to other neurons through synapses whose strengths change with learning. An artificial neuron is a mathematical simplification: it multiplies each input by a weight, adds them with a bias and passes the total through an activation function to produce its output.

In simple words

Think of a neuron as a committee member who listens to several advisers. Some advisers they trust a lot (big weights), some hardly at all (small weights), and some they tend to disagree with (negative weights). If the overall advice is strong enough, they speak up.

Overview

  • A biological neuron receives signals through dendrites, combines them in the cell body and fires along its axon when a threshold is reached.
  • The human brain has about 86 billion neurons linked by trillions of synapses.
  • An artificial neuron multiplies inputs by weights, adds a bias and applies an activation function.
  • Warren McCulloch and Walter Pitts proposed the first mathematical neuron in 1943.
  • Artificial neurons are loosely inspired by biology; real neurons are far more complex.
  • Learning changes the weights, much as experience strengthens or weakens synapses.

Why it matters

Every deep-learning model, from image classifiers to large language models, is built from this simple unit repeated millions or billions of times. Knowing what one neuron computes, and how loosely it resembles a real one, grounds everything else in this module.

Key terms

Dendrite, axon, synapse
A real neuron's input branches, output fibre and connection points with other neurons.
Weight
A number scaling one input; learning adjusts weights, as synapses strengthen or weaken.
Bias
A number added to the weighted sum, shifting how easily the neuron activates.
Weighted sum
z = w₁x₁ + w₂x₂ + … + b: the neuron's combined input.
Activation function
A non-linear function applied to z to give the neuron's output.
Firing
A biological neuron sending a spike once its input passes a threshold.

Types

Neurons come in several types, both in the brain and in artificial networks. Comparing them shows what the artificial version keeps and what it leaves out.

1. Biological neurons

Living cells that transmit electrical and chemical signals.

  • How it works: Sensory neurons carry signals from the senses, motor neurons drive muscles and interneurons connect them; a neuron fires an all-or-nothing spike when its inputs pass a threshold.
  • Example: Touching a hot pan: sensory neurons signal the spinal cord, which triggers motor neurons to pull the hand back.
  • When to use it: Study them to understand where neural-network ideas came from, and why real brains learn from far fewer examples.

2. McCulloch-Pitts (threshold) neuron

The first artificial neuron (1943): binary inputs, fixed weights and a threshold.

  • How it works: It outputs 1 if the weighted sum reaches the threshold and 0 otherwise; it can compute logic functions such as AND and OR but cannot learn.
  • Example: With weights 1 and 1 and threshold 2, the neuron computes AND.
  • When to use it: Use it to understand how neurons can represent logic, the starting point of the field.

3. Perceptron

Frank Rosenblatt's learning neuron (1958) with adjustable weights.

  • How it works: It predicts with a step function and updates weights when it makes a mistake, which works for linearly separable data.
  • Example: Learning to separate two kinds of shapes from pixel inputs on the Mark I Perceptron machine.
  • When to use it: Use it to see how a neuron learns from errors, the idea behind all later training.

4. Modern artificial neurons

Neurons with smooth or piecewise activations such as sigmoid, tanh and ReLU.

  • How it works: Because the activation has useful gradients, the weights can be trained by backpropagation in networks of many layers.
  • Example: A neuron computing ReLU(0.6 × 0.5 − 0.4 × 0.8 + 0.1) = ReLU(0.08) = 0.08.
  • When to use it: Use them as the building blocks of every deep-learning model today.

5. Spiking neurons and neuromorphic hardware

Artificial neurons that communicate with timed spikes, closer to biology.

  • How it works: Information is carried by when spikes happen; neuromorphic chips such as Intel's Loihi run them using very little energy.
  • Example: Low-power sensors that recognise sounds using spiking networks.
  • When to use it: Watch this research area for energy-efficient AI on small devices.

Explained in detail

The biological neuron

Neurons were first described in detail around 1900 by the Spanish scientist Santiago Ramón y Cajal, who shared the 1906 Nobel Prize for this work. Each neuron collects chemical signals from other neurons at its dendrites. If enough excitation arrives within a short time, the neuron fires an electrical spike along its axon, which releases chemicals at synapses onto the next neurons. Learning happens largely by changing synapse strengths; the psychologist Donald Hebb proposed in 1949 that connections between neurons that are active together grow stronger, often summarised as "cells that fire together wire together".

The artificial neuron

In 1943 Warren McCulloch and Walter Pitts proposed a mathematical neuron that added binary inputs and fired if the total passed a threshold. Modern artificial neurons generalise this. Given inputs x₁ to xₙ, the neuron computes the weighted sum z = w₁x₁ + … + wₙxₙ + b and outputs a = f(z), where f is an activation function such as ReLU or sigmoid. For example, with inputs 2 and 3, weights 0.5 and −1, and bias 1, z = 1 − 3 + 1 = −1; ReLU gives 0, so the neuron stays silent; sigmoid gives about 0.27. Training changes the weights and bias so that, across many examples, the network's outputs become correct.

How similar are they?

FeatureBiological neuronArtificial neuron
SignalTimed electrical spikesA single real number
ComputationComplex chemistry and timing within the cellA weighted sum and a simple function
LearningLocal changes at synapses, still not fully understoodGradient descent with backpropagation
ScaleAbout 86 billion neurons, around 100 trillion synapsesLarge models have billions to trillions of weights
EnergyAbout 20 watts for the whole brainTraining a large model uses enormous energy

Artificial neurons were inspired by biology, but modern deep learning has developed largely through mathematics and engineering, not neuroscience. Saying that a network "thinks like a brain" overstates the similarity. The useful lesson from biology is the idea that many simple units, each adjusting its connections, can together represent extraordinarily complex patterns.

A single artificial neuron can only separate classes with a straight line or flat plane. Its real power appears when thousands are connected in layers, so that each neuron's output becomes another neuron's input, which is the subject of the next topics.

How it works

  1. Multiply each input by its weight.
  2. Add the products and the bias to get z.
  3. Apply the activation function to get the output.
  4. Pass the output to the neurons in the next layer.
  5. During training, adjust weights and bias to reduce the loss.

In code

neuron.py
import math

def neuron(inputs, weights, bias, activation):
    z = sum(x * w for x, w in zip(inputs, weights)) + bias
    return z, activation(z)

relu = lambda z: max(0.0, z)
sigmoid = lambda z: 1 / (1 + math.exp(-z))

# Should I go for a run? inputs: sunny (0/1), free hours, tiredness (0-10)
weights, bias = [1.5, 0.8, -0.4], -1.0
for day in [(1, 2, 2), (0, 1, 7), (1, 0.5, 5)]:
    z, out = neuron(day, weights, bias, sigmoid)
    print(f"inputs={day}  z={z:5.2f}  sigmoid={out:.2f}  relu={relu(z):.2f}")

Output

inputs=(1, 2, 2)  z= 1.30  sigmoid=0.79  relu=1.30
inputs=(0, 1, 7)  z=-3.00  sigmoid=0.05  relu=0.00
inputs=(1, 0.5, 5)  z=-1.10  sigmoid=0.25  relu=0.00

Try it yourself

A neuron decides about a run

One artificial neuron decides whether you will go for a run, from three inputs.

Match the biological part to its artificial counterpart

Which artificial idea corresponds to each?

  • Dendrites receiving signals
  • Synapse strength
  • Firing when input is strong enough
  • Axon carrying the signal onward

Common mistakes

  • Believing neural networks work like real brains.
  • Forgetting the bias, which shifts when a neuron activates.
  • Treating a single neuron as able to learn any pattern.

Check yourself

Question 1

1. What does an artificial neuron compute before its activation?

Question 2

2. In a real brain, learning mainly changes…

Question 3

3. Inputs 2 and 3, weights 0.5 and −1, bias 1, ReLU. The output is…

Summary

  • Biological neurons combine signals and fire; learning changes synapse strengths.
  • An artificial neuron computes a weighted sum plus bias, then an activation.
  • The resemblance is loose; the power comes from many simple units working together.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: Goodfellow, Bengio and Courville, Deep Learning (MIT Press) · Dive into Deep Learning (Zhang, Lipton, Li, Smola)