I4 · Problem Solving and Reasoning: Classical AI

Types of Agents

Agents are commonly grouped into five designs of increasing capability: simple reflex agents act on the current percept using condition-action rules; model-based reflex agents keep an internal model of the world to handle what they cannot currently see; goal-based agents choose actions that lead towards a goal; utility-based agents choose actions that maximise a measure of how good outcomes are; and learning agents improve their behaviour from experience.

8 min Intermediate · Sublevel Intermediate Jump to the activity

What you need: The earlier topics of this module.

Definition

Some agents react instantly to what they see; others remember the past; others plan towards goals; others weigh how happy each outcome would make them; the best ones learn. Which design suits which task?

Agents are commonly grouped into five designs of increasing capability: simple reflex agents act on the current percept using condition-action rules; model-based reflex agents keep an internal model of the world to handle what they cannot currently see; goal-based agents choose actions that lead towards a goal; utility-based agents choose actions that maximise a measure of how good outcomes are; and learning agents improve their behaviour from experience.

In simple words

A reflex is pulling your hand from a hot pan. A model is remembering the pan is still hot after you look away. A goal is "cook dinner". Utility is "cook a dinner that is tasty, healthy and ready by eight". Learning is cooking it better next time.

Overview

  • Agents range from simple reflexes to systems that learn and plan.
  • Each type adds a capability: memory, goals, preferences or learning.
  • More capable agents are more flexible but need more computation and knowledge.
  • Real systems often combine types, such as a learning agent with goal-based planning.
  • The classification comes from Russell and Norvig's textbook Artificial Intelligence: A Modern Approach.
  • Choosing the simplest type that works keeps a system reliable and explainable.

Why it matters

Choosing the right agent design avoids both over-engineering simple tasks and under-engineering complex ones. Most real AI systems combine several designs, and recognising them helps you understand how they work and fail.

Key terms

Simple reflex agent
Acts only on the current percept with if-then rules.
Model-based agent
Keeps an internal state describing parts of the world it cannot currently see.
Goal-based agent
Considers the future and chooses actions that achieve a goal.
Utility function
A function scoring how desirable each state or outcome is.
Learning agent
An agent with a learning element that improves its performance from feedback.
Critic
The part of a learning agent that judges how well it is doing.

Types

Russell and Norvig describe five classic types of agent, each more capable than the last.

1. Simple reflex agent

An agent that acts only on the current percept using condition-action rules.

  • How it works: Rules of the form "if dirty, then suck" map what it senses now straight to an action; it has no memory.
  • Example: A thermostat that switches the heater on below 20°C; an automatic door that opens when someone is near.
  • When to use it: Use it for fully observable, simple tasks; it fails when the right action depends on things it cannot see now.

2. Model-based reflex agent

A reflex agent that keeps an internal state, a model of how the world works and changes.

  • How it works: It updates its state from each percept and its knowledge of how actions change the world, then applies rules to that state.
  • Example: A robot vacuum that remembers which rooms it has already cleaned.
  • When to use it: Use it in partially observable environments where remembering the past is enough to act well.

3. Goal-based agent

An agent that chooses actions to reach a stated goal.

  • How it works: It considers future states, often by search or planning, and picks a sequence of actions that ends at a goal state.
  • Example: A navigation app finding a route to the railway station.
  • When to use it: Use it when there is a clear target and several ways to get there.

4. Utility-based agent

An agent that chooses actions that maximise a utility, a number measuring how good an outcome is.

  • How it works: Utility lets it trade off goals, such as speed against safety or cost, and handle uncertainty by maximising expected utility.
  • Example: A ride-sharing app balancing waiting time, distance and price when matching drivers.
  • When to use it: Use it when goals conflict or outcomes are uncertain and some results are better than others.

5. Learning agent

An agent that improves its behaviour from experience.

  • How it works: A learning element changes the performance element using feedback from a critic; a problem generator suggests new actions to explore.
  • Example: A recommendation system that improves as it learns what each user clicks; AlphaGo improving through self-play.
  • When to use it: Use it when the environment is too complex to program by hand or changes over time.

Explained in detail

Five designs compared

DesignHow it decidesExampleWeakness
Simple reflexIf the current percept matches a condition, do the actionAutomatic door, thermostatFails when the right action depends on history or hidden facts
Model-based reflexUpdates an internal model, then applies rulesA car tracking vehicles in its blind spotStill has no explicit goals
Goal-basedSearches or plans for action sequences that reach a goalMaps app finding a routeTreats all goal states as equally good
Utility-basedChooses the action with the highest expected utilityRoute choice trading time, tolls and safetyUtility can be hard to define
LearningImproves any of the above from experienceA recommender improving from clicksCan learn the wrong thing from biased feedback

From goals to utility

A goal says only whether a state is acceptable: you reached Mumbai or you did not. Most real decisions involve trade-offs: arriving in six hours with tolls, or in seven without; a cheap flight with a layover, or a costly direct one. A utility function assigns a number to each outcome so that these can be compared. Under uncertainty, a utility-based agent chooses the action with the highest expected utility, the average utility weighted by the probability of each outcome. This idea, formalised by John von Neumann and Oskar Morgenstern in 1944, links classical AI to economics and to the reward signals of reinforcement learning.

How learning agents are organised

  • The performance element chooses actions, using any of the designs above.
  • The critic compares results with a standard and gives feedback, such as a reward.
  • The learning element uses that feedback to improve the performance element.
  • The problem generator suggests exploratory actions that might teach something new.

Modern systems combine designs. A self-driving car uses reflex rules for emergency braking, a model of nearby vehicles, goals for the destination, utilities to balance speed and comfort, and learning from millions of kilometres of driving. LLM-based assistants that use tools are goal-based agents whose policy was learned from data.

When you read about any AI product, try to place it in this scheme. It quickly reveals what the system can and cannot handle.

How it works

  1. Ask whether the current percept is enough to decide; if so, a reflex agent may do.
  2. If hidden or past information matters, add an internal model.
  3. If actions must be sequenced towards an outcome, make it goal-based with search or planning.
  4. If outcomes involve trade-offs or risk, use a utility function.
  5. If the environment is unknown or changing, add learning, with careful feedback.

In code

utility_agent.py
# A utility-based route chooser: trade off time, toll and risk.
routes = {
    "expressway": {"hours": 6.0, "toll": 800, "risk": 0.02},
    "highway":    {"hours": 7.0, "toll": 200, "risk": 0.03},
    "ghat road":  {"hours": 6.5, "toll": 0,   "risk": 0.08},
}

def utility(r, value_of_hour=300, cost_of_incident=20000):
    return -(r["hours"] * value_of_hour + r["toll"] + r["risk"] * cost_of_incident)

for name, r in routes.items():
    print(f"{name:>10}: utility = {utility(r):8.0f}")
best = max(routes, key=lambda name: utility(routes[name]))
print("Goal-based agent: any route reaches the city. Utility-based choice:", best)

# A traveller who values time much more chooses differently
best_hurry = max(routes, key=lambda name: utility(routes[name], value_of_hour=1500))
print("In a hurry:", best_hurry)

Output

expressway: utility =    -3000
   highway: utility =    -2900
 ghat road: utility =    -3550
Goal-based agent: any route reaches the city. Utility-based choice: highway
In a hurry: expressway

Try it yourself

Five agents, one kitchen

Imagine five robot cooks, each with a different design.

Match the system to its design

Which agent design fits best?

  • An automatic door
  • A car tracking a vehicle in its blind spot
  • A maps app finding any route to the station
  • A trip planner balancing cost, time and comfort
  • A recommender improving from your clicks

Common mistakes

  • Using a reflex design where history or hidden information matters.
  • Treating every goal state as equally good when trade-offs exist.
  • Letting a learning agent learn from biased or easily gamed feedback.

Check yourself

Question 1

1. Which agent keeps track of parts of the world it cannot currently see?

Question 2

2. What does a utility-based agent add over a goal-based one?

Question 3

3. In a learning agent, what does the critic do?

Summary

  • Agent designs grow from reflex to model-based, goal-based, utility-based and learning.
  • Utility handles trade-offs; expected utility handles risk.
  • Real systems combine designs; use the simplest that works.

Finished reading?

Sources: Russell and Norvig, Artificial Intelligence: A Modern Approach (4th edition) · UC Berkeley CS188: Introduction to Artificial Intelligence