I3 · Mathematics for AI

Functions and Graphs

A function is a rule that assigns exactly one output to each input, written y = f(x). Its domain is the set of allowed inputs and its range is the set of outputs it can produce. The graph of a function plots each input x against its output f(x), so the shape of the curve shows how the output changes as the input changes.

8 min Intermediate · Sublevel Beginner Jump to the activity

What you need: The earlier topics of this module.

Definition

A model takes in the area of a house and gives out a price. A neuron takes in a number and gives out a number between 0 and 1. Both are functions, and drawing them tells you how they behave.

A function is a rule that assigns exactly one output to each input, written y = f(x). Its domain is the set of allowed inputs and its range is the set of outputs it can produce. The graph of a function plots each input x against its output f(x), so the shape of the curve shows how the output changes as the input changes.

In simple words

A function is like a vending machine: put in a code, get exactly one item. The same code always gives the same item. A graph is a picture of every code and its item at once, so you can see the pattern instead of trying each one.

Overview

  • A function maps each input to exactly one output: y = f(x).
  • The domain is the set of allowed inputs; the range is the set of outputs.
  • A graph plots every (x, f(x)) pair, showing how the output changes.
  • Slope, intercepts, maxima, minima and asymptotes can be read from a graph.
  • Every machine-learning model is a function from inputs to predictions, shaped by its parameters.
  • Activation and loss functions have particular shapes chosen for good learning.

Why it matters

Every machine-learning model is a function from inputs to predictions, every loss is a function of the parameters, and every activation in a neural network is a small function. Recognising the shapes of linear, quadratic, exponential, logarithmic and sigmoid functions lets you predict how models will behave.

Key terms

Function
A rule giving exactly one output for each input.
Domain
The set of inputs a function accepts.
Range
The set of outputs a function can produce.
Slope and intercept
In y = mx + c, m is the steepness and c is where the line crosses the y-axis.
Composition
Applying one function to the output of another: f(g(x)).
Sigmoid
An S-shaped function, 1 / (1 + e⁻ˣ), squashing any number into the range 0 to 1.

Types

A few families of function appear everywhere in AI. These are the types and where you will see them.

1. Linear functions

Straight-line functions f(x) = mx + c.

  • How it works: m is the slope, the change in y per unit of x, and c is the y-intercept; in many dimensions f(x) = w · x + b.
  • Example: A taxi fare of ₹50 plus ₹15 per km: f(x) = 15x + 50.
  • When to use it: Use them as the simplest models (linear regression) and as the building block inside neurons.

2. Polynomial functions

Sums of powers of x, such as f(x) = ax² + bx + c.

  • How it works: A quadratic is a parabola with one turning point at x = −b ÷ 2a; higher degrees can bend more times.
  • Example: The path of a thrown ball; yield that peaks at an ideal fertiliser dose.
  • When to use it: Use them for curved relationships in polynomial regression, and the squared-error loss is a quadratic.

3. Exponential and logarithmic functions

f(x) = aˣ or eˣ, which grows by a constant factor, and its inverse log x.

  • How it works: Exponentials grow or decay very fast; logarithms grow slowly and turn multiplication into addition: log(ab) = log a + log b.
  • Example: Compound interest and virus spread are exponential; the decibel and Richter scales are logarithmic.
  • When to use it: Use them in softmax and cross-entropy, for log-transforming skewed features and for describing algorithm costs.

4. Sigmoid and other S-shaped functions

Functions that squash any input into a fixed range, such as σ(x) = 1 ÷ (1 + e⁻ˣ) and tanh.

  • How it works: Sigmoid outputs lie between 0 and 1 with σ(0) = 0.5; tanh outputs lie between −1 and 1.
  • Example: Logistic regression turns a score of 2 into a probability of σ(2) ≈ 0.88.
  • When to use it: Use them to output probabilities and as gates inside LSTMs.

5. Piecewise functions such as ReLU

Functions defined by different rules on different parts of the input.

  • How it works: ReLU(x) = max(0, x) is 0 for negative inputs and x otherwise; absolute value and step functions are other examples.
  • Example: Income-tax slabs are piecewise; ReLU(−3) = 0 and ReLU(2) = 2.
  • When to use it: Use ReLU as the default activation in deep networks; piecewise rules also model slabs and thresholds.

Explained in detail

Families of functions

FamilyExampleShapeIn AI
Lineary = 2x + 1Straight lineLinear regression; the weighted sum inside a neuron
Quadraticy = x²U-shaped bowlSquared-error loss, which gradient descent rolls down
Exponentialy = 2ˣ or eˣSlow, then explosive growthSoftmax; compound growth; the cost of brute-force search
Logarithmicy = log xFast, then slowing growthLog loss; taming skewed features; binary search cost
Sigmoidy = 1 / (1 + e⁻ˣ)S-curve from 0 to 1Turning scores into probabilities in logistic regression
ReLUy = max(0, x)Flat at 0, then a straight lineThe most common activation in deep networks

Reading a graph

  • Where does it cross the axes? The y-intercept is f(0); the x-intercepts are the inputs that give 0.
  • Is it increasing, decreasing or both? A loss curve should mostly decrease during training.
  • Where is it steep and where is it flat? The steepness is the derivative.
  • Does it level off? The sigmoid flattens near 0 and 1, which is why its gradients become tiny there.
  • Where is the lowest or highest point? Training searches for the lowest point of the loss.

Composition: how networks are built

Composing functions means feeding the output of one into another. If g(x) = 2x + 1 and f(x) = max(0, x), then f(g(x)) first scales and shifts the input, then cuts off negative values. A neural network is exactly a long composition: each layer is a linear function followed by an activation, and the output of one layer is the input of the next. Without the non-linear activations, composing linear functions would produce just another linear function, however many layers you stacked. The bends introduced by ReLU or sigmoid are what let deep networks model curved, complicated relationships.

Transformations

Small changes to a formula move its graph in predictable ways. f(x) + c shifts it up by c; f(x − c) shifts it right by c; a × f(x) stretches it vertically; f(ax) squeezes it horizontally. In a neuron, the weight stretches or flips the activation and the bias shifts it left or right, which is how training positions each neuron's response where it is useful.

How it works

  1. Identify the input, the output and the rule.
  2. Work out the domain: are any inputs forbidden, such as dividing by zero or the log of a negative?
  3. Compute a few points, including 0, negative values and large values.
  4. Plot them and recognise the family.
  5. Ask what the shape means for your problem: where is it steep, flat, lowest?

The maths

Straight line
The slope
The y-intercept
Sigmoid
ReLU

In code

functions.py
import math

def linear(x):
    return 2 * x + 1

def sigmoid(x):
    return 1 / (1 + math.exp(-x))

def relu(x):
    return max(0, x)

print(" x   linear  sigmoid  relu")
for x in [-4, -2, 0, 2, 4]:
    print(f"{x:>2}  {linear(x):>6}  {sigmoid(x):>7.3f}  {relu(x):>4}")

# Composition: a tiny "neuron" is an activation applied to a linear function
neuron = lambda x: relu(linear(x))
print("relu(linear(-3)) =", neuron(-3), "  relu(linear(3)) =", neuron(3))

Output

 x   linear  sigmoid  relu
-4      -7    0.018     0
-2      -3    0.119     0
 0       1    0.500     0
 2       5    0.881     2
 4       9    0.982     4
relu(linear(-3)) = 0   relu(linear(3)) = 7

Try it yourself

Building a neuron from two functions

A learner wants to understand what one neuron does to its input.

Match the function to its role in AI

Which family plays each role?

  • Turns a score into a probability between 0 and 1
  • The bowl-shaped squared-error loss
  • The common activation that zeroes negatives
  • The prediction of simple regression

Common mistakes

  • Feeding inputs outside the domain, such as log(0), and getting errors or infinities.
  • Assuming stacked linear layers can model curves without non-linear activations.
  • Reading a graph's shape without checking the axis scales.

Check yourself

Question 1

1. What does the sigmoid function output?

Question 2

2. Why do neural networks need non-linear activations?

Question 3

3. In y = mx + c, what does c tell you?

Summary

  • A function maps each input to exactly one output; its graph shows the behaviour.
  • Linear, quadratic, exponential, log, sigmoid and ReLU shapes appear throughout AI.
  • Neural networks are compositions of linear functions and non-linear activations.

Finished reading?

Sources: Khan Academy: Functions · Mathematics for Machine Learning (Deisenroth, Faisal and Ong)