I4 · Problem Solving and Reasoning: Classical AI

Agents and Environments

An agent is anything that perceives its environment through sensors and acts upon it through actuators. Its behaviour is described by an agent function mapping the sequence of percepts it has received to an action. A rational agent chooses the action expected to maximise its performance measure, given what it has perceived and what it knows.

10 min Intermediate · Sublevel Intermediate Jump to the activity

What you need: The earlier topics of this module.

Definition

A robot vacuum, a chess program, a self-driving car and a spam filter look nothing alike. Classical AI describes all of them with the same idea: an agent that perceives its environment and acts on it.

An agent is anything that perceives its environment through sensors and acts upon it through actuators. Its behaviour is described by an agent function mapping the sequence of percepts it has received to an action. A rational agent chooses the action expected to maximise its performance measure, given what it has perceived and what it knows.

In simple words

A thermostat is a tiny agent: it senses the temperature (percept), decides whether to switch the heater on (agent function) and flips the switch (action). Its goal is a comfortable room (performance measure). Bigger AI systems follow the same loop, just with far more complex sensing and deciding.

Overview

  • An agent is anything that perceives its environment through sensors and acts on it through actuators.
  • The agent function maps the history of percepts to an action; the agent program implements it.
  • A rational agent chooses the action expected to maximise its performance measure.
  • Tasks are described with PEAS: Performance measure, Environment, Actuators and Sensors.
  • The type of environment decides how hard the task is and which techniques fit.
  • This framework, popularised by Russell and Norvig, describes everything from thermostats to self-driving cars.

Why it matters

The agent view gives one framework for designing and comparing every kind of AI system. Describing a problem by its performance measure, environment, actuators and sensors clarifies what the system must do before any algorithm is chosen.

Key terms

Agent
Something that perceives its environment and acts on it.
Percept
What the agent senses at one moment.
Actuator
The means by which an agent acts: wheels, a display, a message.
Rationality
Choosing the action expected to do best on the performance measure, given what is known.
PEAS
Performance measure, Environment, Actuators, Sensors: a way to specify an agent's task.
Environment properties
Observable or partial, deterministic or stochastic, static or dynamic, discrete or continuous, single or multi-agent.

Types

Environments are classified along several dimensions. Each pair of types changes what the agent must be able to do.

1. Fully versus partially observable

In a fully observable environment the sensors show the whole relevant state; in a partially observable one some is hidden.

  • How it works: With partial observability the agent must keep an internal state or belief about what it cannot see.
  • Example: Chess is fully observable; poker and driving in fog are partially observable.
  • When to use it: Expect to need memory, probability and sensor fusion whenever the environment is partially observable.

2. Deterministic versus stochastic

In a deterministic environment the next state follows exactly from the current state and action; in a stochastic one outcomes are uncertain.

  • How it works: Stochastic environments need probabilities of outcomes and plans that cope with surprises.
  • Example: A puzzle like the 8-puzzle is deterministic; a delivery robot facing traffic and weather is stochastic.
  • When to use it: Use search and planning for deterministic tasks, and probabilistic reasoning or reinforcement learning for stochastic ones.

3. Episodic versus sequential

In an episodic environment each decision is independent; in a sequential one current actions affect future ones.

  • How it works: Sequential tasks require planning ahead, because a move that looks good now can cause trouble later.
  • Example: Classifying each photo on an assembly line is episodic; playing a game or managing a farm season is sequential.
  • When to use it: Use simple classifiers for episodic tasks and search, planning or RL for sequential ones.

4. Static versus dynamic

A static environment does not change while the agent thinks; a dynamic one keeps changing.

  • How it works: In dynamic environments, slow decisions are themselves bad decisions, so agents need time limits and fast reactions.
  • Example: A crossword is static; traffic and stock markets are dynamic.
  • When to use it: Design for speed and continual re-planning when the environment is dynamic.

5. Discrete versus continuous, and single- versus multi-agent

Discrete environments have a finite number of distinct states and actions; continuous ones have smooth ranges. Multi-agent environments contain other agents that cooperate or compete.

  • How it works: Continuous problems are often discretised or solved with calculus and control theory; multi-agent problems need game theory or coordination.
  • Example: Chess is discrete and two-agent; steering a drone is continuous; a team of warehouse robots is cooperative multi-agent.
  • When to use it: Identify these properties first, because they decide which algorithms in this module apply.

Explained in detail

Describing a task with PEAS

AgentPerformanceEnvironmentActuatorsSensors
Self-driving taxiSafe, fast, legal, comfortable tripsRoads, traffic, pedestrians, weatherSteering, brakes, accelerator, hornCameras, lidar, GPS, speedometer
Medical diagnosis assistantCorrect diagnoses, few unnecessary testsPatients, records, hospitalSuggested questions, test orders, reportsSymptoms entered, test results
Spam filterSpam blocked, real mail deliveredEmail stream, usersMove to spam, deliverEmail text, sender, links
Chess programWin, draw over lossBoard, opponentMovesBoard position

Kinds of environment

Environments differ in ways that decide how hard a problem is. In a fully observable environment, such as chess, the agent sees everything relevant; in a partially observable one, such as poker or driving in fog, it must reason about what it cannot see. Deterministic environments always give the same result for an action; stochastic ones, such as dice games or traffic, involve chance. Static environments wait while the agent thinks; dynamic ones change, so slow thinking is costly. Discrete environments have a countable number of states and actions; continuous ones, such as steering angles, do not. Single-agent problems differ from multi-agent ones, where other agents may cooperate or compete. Real-world tasks such as driving are usually the hardest kind on every dimension.

Rational is not the same as perfect

  • A rational agent does the best it can with what it knows; it cannot know the future.
  • Gathering information, such as looking both ways before crossing, is often the rational action.
  • Rationality depends on the performance measure: a taxi judged only on speed will drive dangerously.
  • Choosing the performance measure is therefore an ethical decision as much as a technical one.

This framework, popularised by Stuart Russell and Peter Norvig's textbook Artificial Intelligence: A Modern Approach, first published in 1995 and used in over 1,500 universities, organises the rest of this module: search, games, constraints, logic, planning and reasoning under uncertainty are all ways of building the agent function for different kinds of environment.

The same vocabulary applies to modern AI. A chatbot is an agent whose percepts are messages and whose actions are replies; a recommendation system perceives clicks and acts by choosing what to show. Thinking in terms of agents and environments helps you ask the right questions: what does the system observe, what can it do, and what exactly is it trying to achieve?

How it works

  1. Name the agent and write its performance measure first.
  2. Describe the environment and its properties.
  3. List the actuators and sensors.
  4. Decide what kind of agent design suits these properties.
  5. Check the performance measure cannot be satisfied in harmful ways.

In code

vacuum_agent.py
# A tiny reflex agent in a two-room world: rooms A and B, each Clean or Dirty.
world = {"A": "Dirty", "B": "Dirty"}
location = "A"
score = 0

def agent(percept):
    room, status = percept
    if status == "Dirty":
        return "Suck"
    return "Right" if room == "A" else "Left"

for step in range(6):
    percept = (location, world[location])
    action = agent(percept)
    if action == "Suck":
        world[location] = "Clean"
    else:
        location = "B" if action == "Right" else "A"
    score += sum(status == "Clean" for status in world.values())  # reward clean rooms each step
    print(f"step {step}: percept={percept} action={action} world={world}")
print("Performance score:", score)

Output

step 0: percept=('A', 'Dirty') action=Suck world={'A': 'Clean', 'B': 'Dirty'}
step 1: percept=('A', 'Clean') action=Right world={'A': 'Clean', 'B': 'Dirty'}
step 2: percept=('B', 'Dirty') action=Suck world={'A': 'Clean', 'B': 'Clean'}
step 3: percept=('B', 'Clean') action=Left world={'A': 'Clean', 'B': 'Clean'}
step 4: percept=('A', 'Clean') action=Right world={'A': 'Clean', 'B': 'Clean'}
step 5: percept=('B', 'Clean') action=Left world={'A': 'Clean', 'B': 'Clean'}
Performance score: 10

Try it yourself

The taxi that only cared about speed

Engineers design a self-driving taxi and must choose its performance measure.

Classify the environment

Is each environment fully or partially observable?

  • Chess

  • Driving in fog

  • Poker

  • A sliding-tile puzzle

Common mistakes

  • Choosing a performance measure that can be gamed in harmful ways.
  • Assuming the environment is fully observable and deterministic when it is not.
  • Confusing rational with all-knowing.

Check yourself

Question 1

1. What does PEAS stand for?

Question 2

2. Poker is…

Question 3

3. A rational agent…

Summary

  • An agent perceives through sensors and acts through actuators.
  • PEAS and environment properties define the task before choosing an algorithm.
  • Rationality depends on the performance measure, which must be chosen carefully.

Read it. Now try it yourself.

Open the interactive activity

Finished reading?

Sources: Russell and Norvig, Artificial Intelligence: A Modern Approach (4th edition) · UC Berkeley CS188: Introduction to Artificial Intelligence