B5 · Generative AI and Conversational Assistants

Types of Generative AI

Generative AI tools are grouped by what they produce: text, images, audio (speech and music), video, and computer programs. A tool that can take in or produce more than one of these is called multimodal.

6 min Beginner Jump to the activity

What you need: Nothing. This page starts from the beginning, with no programming and no formulas.

Definition

A story, a painting, a song, a film clip and a working app: can one kind of technology really make all five?

Generative AI tools are grouped by what they produce: text, images, audio (speech and music), video, and computer programs. A tool that can take in or produce more than one of these is called multimodal.

In simple words

Think of an art room with different tables. At one table people write, at another they paint, at a third they compose music, at a fourth they make short films. Generative AI has a tool for each table. Some newer tools can walk between tables: you give them a photo and they write about it, or you give them words and they paint.

Why it matters

Each type has its own strengths, its own typical mistakes and its own risks. A wrong sentence, a fake photograph and a cloned voice cause different kinds of harm. Knowing the types helps you choose the right tool and know what to check.

Key terms

Generative AI
AI that produces new content, not only a judgement about existing content.
Text generator
A tool that writes text, such as answers, stories, summaries and translations.
Image generator
A tool that makes pictures from a written description.
Synthetic voice
Speech produced by a computer, sometimes copying a real person's voice.
Multimodal
Able to work with more than one kind of content, such as text and images together.
Prompt
The instruction you give to a generative AI tool.

Explained in detail

Text

Text generators are the most widely used type. They answer questions, explain ideas, write letters and stories, summarise long documents, translate between languages and hold conversations. The chat assistants that became popular from the end of 2022 belong here. Their strength is fluent language on almost any subject. Their characteristic weakness is that fluent does not mean true: they can state invented facts in perfect sentences.

Images

Image generators turn a description such as "a tea stall in the rain, watercolour style" into a picture in seconds. Tools of this kind reached the public in 2022. They can produce photographs that never happened, illustrations in any style, logos and designs. They can also edit an existing picture: remove an object, extend the edges, change day into night. Early versions struggled with hands, with written words inside the picture and with consistent details. These faults are becoming rarer, which makes generated images harder to recognise.

Audio: speech and music

Speech generators read text aloud in a natural voice, in many languages. They power audiobooks, announcements and screen readers for blind users. With a short recording they can also imitate a particular person's voice, which is useful for someone who is losing the ability to speak and dangerous in the hands of a fraudster. Music generators compose a tune, or a whole song with singing, from a description such as "a cheerful folk tune with a flute".

Video

Video generators are the newest and the most demanding to build, because a video is many images that must stay consistent from one frame to the next: the same face, the same clothes, objects that move in believable ways. Tools shown from 2024 onwards can produce short, realistic clips from a sentence. Longer videos with a clear story are still difficult. Realistic generated video raises the most serious worries about deception, because people have long treated video as proof.

Computer programs

Programs are written in special languages with strict rules, and language models can write in those too. A person describes what a program should do, and the tool writes it, explains it or finds mistakes in it. People with no training can now build a simple app or website this way. For anything important, a person who understands programming still needs to check the result, because a program that looks right can contain hidden errors or security holes.

Multimodal tools

The boundaries between these types are fading. Many assistants now accept a photo, a document, a voice message or a video along with your question, and reply in text, speech or pictures. You can photograph a broken appliance and ask what is wrong, or sketch a web page on paper and ask for a working version. One tool increasingly covers all the tables in the art room.

How it works

TypeYou giveYou getCheck for
TextA question or instructionWritten answer, story, summaryInvented facts and sources
ImageA descriptionA pictureFake "photos" of real events or people
SpeechText, and sometimes a voice sampleA spoken recordingCloned voices used in scams
MusicA description of mood and styleA tune or songClose copies of existing songs
VideoA description or an imageA short clipEvents that never happened
ProgramsA description of what it should doA working programHidden errors and unsafe behaviour

Try it yourself

One idea, five kinds of output

A group of friends is preparing for the school's annual day. Watch them use a different type of tool at each step.

Match the tool to what it makes

Which type of generative AI would you use for each job?

  • A birthday invitation with a cartoon elephant
  • A simpler explanation of how rain forms
  • A recording of a story read aloud for a younger sister
  • A calm tune for a yoga video
  • A small quiz game that runs in a web browser

Common mistakes

  • Thinking generative AI means only chatbots. Images, voices, music, video and programs are generated too.
  • Treating a realistic generated image or video as evidence that something happened.
  • Using a generated program for something important without anyone checking it.

Check yourself

Question 1

1. Which tool would turn the words "a red kite over a green field" into a picture?

Question 2

2. What does "multimodal" mean?

Question 3

3. Why is generated video a special concern?

Summary

  • Generative AI produces text, images, speech, music, video and programs.
  • Multimodal tools combine several of these in one assistant.
  • Each type has its own typical faults, so each needs its own kind of checking.

Finished reading?

Sources: UNESCO: Guidance for generative AI in education and research · Elements of AI (University of Helsinki)