Definition
A model takes in the area of a house and gives out a price. A neuron takes in a number and gives out a number between 0 and 1. Both are functions, and drawing them tells you how they behave.
A function is a rule that assigns exactly one output to each input, written y = f(x). Its domain is the set of allowed inputs and its range is the set of outputs it can produce. The graph of a function plots each input x against its output f(x), so the shape of the curve shows how the output changes as the input changes.
In simple words
A function is like a vending machine: put in a code, get exactly one item. The same code always gives the same item. A graph is a picture of every code and its item at once, so you can see the pattern instead of trying each one.
Overview
- A function maps each input to exactly one output: y = f(x).
- The domain is the set of allowed inputs; the range is the set of outputs.
- A graph plots every (x, f(x)) pair, showing how the output changes.
- Slope, intercepts, maxima, minima and asymptotes can be read from a graph.
- Every machine-learning model is a function from inputs to predictions, shaped by its parameters.
- Activation and loss functions have particular shapes chosen for good learning.
Why it matters
Every machine-learning model is a function from inputs to predictions, every loss is a function of the parameters, and every activation in a neural network is a small function. Recognising the shapes of linear, quadratic, exponential, logarithmic and sigmoid functions lets you predict how models will behave.
Key terms
- Function
- A rule giving exactly one output for each input.
- Domain
- The set of inputs a function accepts.
- Range
- The set of outputs a function can produce.
- Slope and intercept
- In y = mx + c, m is the steepness and c is where the line crosses the y-axis.
- Composition
- Applying one function to the output of another: f(g(x)).
- Sigmoid
- An S-shaped function, 1 / (1 + e⁻ˣ), squashing any number into the range 0 to 1.
Types
A few families of function appear everywhere in AI. These are the types and where you will see them.
1. Linear functions
Straight-line functions f(x) = mx + c.
- How it works: m is the slope, the change in y per unit of x, and c is the y-intercept; in many dimensions f(x) = w · x + b.
- Example: A taxi fare of ₹50 plus ₹15 per km: f(x) = 15x + 50.
- When to use it: Use them as the simplest models (linear regression) and as the building block inside neurons.
2. Polynomial functions
Sums of powers of x, such as f(x) = ax² + bx + c.
- How it works: A quadratic is a parabola with one turning point at x = −b ÷ 2a; higher degrees can bend more times.
- Example: The path of a thrown ball; yield that peaks at an ideal fertiliser dose.
- When to use it: Use them for curved relationships in polynomial regression, and the squared-error loss is a quadratic.
3. Exponential and logarithmic functions
f(x) = aˣ or eˣ, which grows by a constant factor, and its inverse log x.
- How it works: Exponentials grow or decay very fast; logarithms grow slowly and turn multiplication into addition: log(ab) = log a + log b.
- Example: Compound interest and virus spread are exponential; the decibel and Richter scales are logarithmic.
- When to use it: Use them in softmax and cross-entropy, for log-transforming skewed features and for describing algorithm costs.
4. Sigmoid and other S-shaped functions
Functions that squash any input into a fixed range, such as σ(x) = 1 ÷ (1 + e⁻ˣ) and tanh.
- How it works: Sigmoid outputs lie between 0 and 1 with σ(0) = 0.5; tanh outputs lie between −1 and 1.
- Example: Logistic regression turns a score of 2 into a probability of σ(2) ≈ 0.88.
- When to use it: Use them to output probabilities and as gates inside LSTMs.
5. Piecewise functions such as ReLU
Functions defined by different rules on different parts of the input.
- How it works: ReLU(x) = max(0, x) is 0 for negative inputs and x otherwise; absolute value and step functions are other examples.
- Example: Income-tax slabs are piecewise; ReLU(−3) = 0 and ReLU(2) = 2.
- When to use it: Use ReLU as the default activation in deep networks; piecewise rules also model slabs and thresholds.
Explained in detail
Families of functions
| Family | Example | Shape | In AI |
|---|---|---|---|
| Linear | y = 2x + 1 | Straight line | Linear regression; the weighted sum inside a neuron |
| Quadratic | y = x² | U-shaped bowl | Squared-error loss, which gradient descent rolls down |
| Exponential | y = 2ˣ or eˣ | Slow, then explosive growth | Softmax; compound growth; the cost of brute-force search |
| Logarithmic | y = log x | Fast, then slowing growth | Log loss; taming skewed features; binary search cost |
| Sigmoid | y = 1 / (1 + e⁻ˣ) | S-curve from 0 to 1 | Turning scores into probabilities in logistic regression |
| ReLU | y = max(0, x) | Flat at 0, then a straight line | The most common activation in deep networks |
Reading a graph
- Where does it cross the axes? The y-intercept is f(0); the x-intercepts are the inputs that give 0.
- Is it increasing, decreasing or both? A loss curve should mostly decrease during training.
- Where is it steep and where is it flat? The steepness is the derivative.
- Does it level off? The sigmoid flattens near 0 and 1, which is why its gradients become tiny there.
- Where is the lowest or highest point? Training searches for the lowest point of the loss.
Composition: how networks are built
Composing functions means feeding the output of one into another. If g(x) = 2x + 1 and f(x) = max(0, x), then f(g(x)) first scales and shifts the input, then cuts off negative values. A neural network is exactly a long composition: each layer is a linear function followed by an activation, and the output of one layer is the input of the next. Without the non-linear activations, composing linear functions would produce just another linear function, however many layers you stacked. The bends introduced by ReLU or sigmoid are what let deep networks model curved, complicated relationships.
Transformations
Small changes to a formula move its graph in predictable ways. f(x) + c shifts it up by c; f(x − c) shifts it right by c; a × f(x) stretches it vertically; f(ax) squeezes it horizontally. In a neuron, the weight stretches or flips the activation and the bias shifts it left or right, which is how training positions each neuron's response where it is useful.
How it works
- Identify the input, the output and the rule.
- Work out the domain: are any inputs forbidden, such as dividing by zero or the log of a negative?
- Compute a few points, including 0, negative values and large values.
- Plot them and recognise the family.
- Ask what the shape means for your problem: where is it steep, flat, lowest?
The maths
- The slope
- The y-intercept
In code
import math
def linear(x):
return 2 * x + 1
def sigmoid(x):
return 1 / (1 + math.exp(-x))
def relu(x):
return max(0, x)
print(" x linear sigmoid relu")
for x in [-4, -2, 0, 2, 4]:
print(f"{x:>2} {linear(x):>6} {sigmoid(x):>7.3f} {relu(x):>4}")
# Composition: a tiny "neuron" is an activation applied to a linear function
neuron = lambda x: relu(linear(x))
print("relu(linear(-3)) =", neuron(-3), " relu(linear(3)) =", neuron(3))Output
x linear sigmoid relu -4 -7 0.018 0 -2 -3 0.119 0 0 1 0.500 0 2 5 0.881 2 4 9 0.982 4 relu(linear(-3)) = 0 relu(linear(3)) = 7
Try it yourself
Building a neuron from two functions
A learner wants to understand what one neuron does to its input.
Match the function to its role in AI
Which family plays each role?
- Turns a score into a probability between 0 and 1
- The bowl-shaped squared-error loss
- The common activation that zeroes negatives
- The prediction of simple regression
Common mistakes
- Feeding inputs outside the domain, such as log(0), and getting errors or infinities.
- Assuming stacked linear layers can model curves without non-linear activations.
- Reading a graph's shape without checking the axis scales.
Check yourself
Summary
- A function maps each input to exactly one output; its graph shows the behaviour.
- Linear, quadratic, exponential, log, sigmoid and ReLU shapes appear throughout AI.
- Neural networks are compositions of linear functions and non-linear activations.
Finished reading?
Sources: Khan Academy: Functions · Mathematics for Machine Learning (Deisenroth, Faisal and Ong)