Definition
A story, a painting, a song, a film clip and a working app: can one kind of technology really make all five?
Generative AI tools are grouped by what they produce: text, images, audio (speech and music), video, and computer programs. A tool that can take in or produce more than one of these is called multimodal.
In simple words
Think of an art room with different tables. At one table people write, at another they paint, at a third they compose music, at a fourth they make short films. Generative AI has a tool for each table. Some newer tools can walk between tables: you give them a photo and they write about it, or you give them words and they paint.
Why it matters
Each type has its own strengths, its own typical mistakes and its own risks. A wrong sentence, a fake photograph and a cloned voice cause different kinds of harm. Knowing the types helps you choose the right tool and know what to check.
Key terms
- Generative AI
- AI that produces new content, not only a judgement about existing content.
- Text generator
- A tool that writes text, such as answers, stories, summaries and translations.
- Image generator
- A tool that makes pictures from a written description.
- Synthetic voice
- Speech produced by a computer, sometimes copying a real person's voice.
- Multimodal
- Able to work with more than one kind of content, such as text and images together.
- Prompt
- The instruction you give to a generative AI tool.
Explained in detail
Text
Text generators are the most widely used type. They answer questions, explain ideas, write letters and stories, summarise long documents, translate between languages and hold conversations. The chat assistants that became popular from the end of 2022 belong here. Their strength is fluent language on almost any subject. Their characteristic weakness is that fluent does not mean true: they can state invented facts in perfect sentences.
Images
Image generators turn a description such as "a tea stall in the rain, watercolour style" into a picture in seconds. Tools of this kind reached the public in 2022. They can produce photographs that never happened, illustrations in any style, logos and designs. They can also edit an existing picture: remove an object, extend the edges, change day into night. Early versions struggled with hands, with written words inside the picture and with consistent details. These faults are becoming rarer, which makes generated images harder to recognise.
Audio: speech and music
Speech generators read text aloud in a natural voice, in many languages. They power audiobooks, announcements and screen readers for blind users. With a short recording they can also imitate a particular person's voice, which is useful for someone who is losing the ability to speak and dangerous in the hands of a fraudster. Music generators compose a tune, or a whole song with singing, from a description such as "a cheerful folk tune with a flute".
Video
Video generators are the newest and the most demanding to build, because a video is many images that must stay consistent from one frame to the next: the same face, the same clothes, objects that move in believable ways. Tools shown from 2024 onwards can produce short, realistic clips from a sentence. Longer videos with a clear story are still difficult. Realistic generated video raises the most serious worries about deception, because people have long treated video as proof.
Computer programs
Programs are written in special languages with strict rules, and language models can write in those too. A person describes what a program should do, and the tool writes it, explains it or finds mistakes in it. People with no training can now build a simple app or website this way. For anything important, a person who understands programming still needs to check the result, because a program that looks right can contain hidden errors or security holes.
Multimodal tools
The boundaries between these types are fading. Many assistants now accept a photo, a document, a voice message or a video along with your question, and reply in text, speech or pictures. You can photograph a broken appliance and ask what is wrong, or sketch a web page on paper and ask for a working version. One tool increasingly covers all the tables in the art room.
How it works
| Type | You give | You get | Check for |
|---|---|---|---|
| Text | A question or instruction | Written answer, story, summary | Invented facts and sources |
| Image | A description | A picture | Fake "photos" of real events or people |
| Speech | Text, and sometimes a voice sample | A spoken recording | Cloned voices used in scams |
| Music | A description of mood and style | A tune or song | Close copies of existing songs |
| Video | A description or an image | A short clip | Events that never happened |
| Programs | A description of what it should do | A working program | Hidden errors and unsafe behaviour |
Try it yourself
One idea, five kinds of output
A group of friends is preparing for the school's annual day. Watch them use a different type of tool at each step.
Match the tool to what it makes
Which type of generative AI would you use for each job?
- A birthday invitation with a cartoon elephant
- A simpler explanation of how rain forms
- A recording of a story read aloud for a younger sister
- A calm tune for a yoga video
- A small quiz game that runs in a web browser
Common mistakes
- Thinking generative AI means only chatbots. Images, voices, music, video and programs are generated too.
- Treating a realistic generated image or video as evidence that something happened.
- Using a generated program for something important without anyone checking it.
Check yourself
Summary
- Generative AI produces text, images, speech, music, video and programs.
- Multimodal tools combine several of these in one assistant.
- Each type has its own typical faults, so each needs its own kind of checking.
Finished reading?
Sources: UNESCO: Guidance for generative AI in education and research · Elements of AI (University of Helsinki)