Definition
When an AI tool writes a poem, is it copying one from somewhere, or making it up?
Generative AI creates content in two stages. In training, a model studies an enormous collection of examples and learns the patterns in them. In generation, it uses those patterns to build something new, one small step at a time, guided by your prompt.
In simple words
A cook who has eaten and studied thousands of dishes can invent a new recipe. The new dish is not a copy of any one meal, but everything in it comes from what the cook has tasted before. A generative model is like that cook. It has absorbed patterns from millions of examples, and it combines them into something that did not exist until you asked.
Why it matters
Understanding the process answers the questions people most often ask: why the same prompt gives different results, why the output can be wrong, and whether the tool is copying. It replaces the idea of magic with a mechanism you can reason about.
Key terms
- Training data
- The collection of text, images or sound a model learns from.
- Pattern
- A regularity in the data, such as which words tend to follow which.
- Generation
- Producing new content from a trained model.
- Next-token prediction
- Choosing the next small piece of text, given everything written so far.
- Noise
- Random dots with no pattern. Image generators start from noise.
- Randomness
- A controlled element of chance that makes each output slightly different.
Explained in detail
Stage one: learning patterns
Before anyone can use a generative model, it is trained. A text model is given an enormous amount of writing: books, articles, websites, conversations. An image model is given hundreds of millions of pictures, each with a caption. The model does not file these away to look up later. It adjusts its billions of internal numbers so that it becomes good at one task: predicting what comes next, or what is missing. In doing so it absorbs grammar, facts, styles of writing, the look of a sunset, the shape of a hand.
Stage two, for text: one piece at a time
When you send a prompt, a text model does something surprisingly simple. It looks at all the text so far and works out, for every piece of text in its vocabulary, how likely that piece is to come next. After "The capital of India is", the piece "New" is very likely and "banana" is not. It picks one, adds it to the text, and repeats the whole calculation for the next piece. A paragraph of a hundred words is the result of well over a hundred such steps. The model never plans the paragraph in advance; the text grows from left to right.
Stage two, for images: from static to picture
Most image generators work by a method called diffusion. In training, the model is shown a picture with a little random noise added, like static on an old television, and learns to predict what noise was added. This is repeated with more and more noise, until the model can find a picture even in something that looks like pure static. To generate, the process runs in reverse. The model starts with a square of pure noise and removes a little of it, step by step, each time steering towards an image that matches your description. After a few dozen steps a clear picture has emerged from nothing.
Why the same prompt gives different results
If a text model always picked the single most likely next piece, it would give the same answer every time, and its writing would be dull and repetitive. So it is allowed a measured amount of randomness: it usually picks a likely piece, and sometimes a slightly less likely one. An image model begins from a different patch of random noise each time. This is why asking twice gives two different poems or two different pictures. It is also why you should not treat any single answer as the tool's fixed view.
Is it copying?
Mostly, no. The model does not hold a library of its training examples and paste pieces together. It holds patterns, and builds each output afresh. But the honest answer has two further parts. First, everything it produces is shaped by the work of the people whose writing and pictures it learned from, usually without their being asked. Second, a model can sometimes reproduce something very close to a training example, particularly a famous text or image that appeared many times in its data. That is one reason the law around generative AI is still being argued in courts.
What it does not do
- It does not look facts up in a database, unless the product has been connected to a search tool.
- It does not check whether what it wrote is true.
- It does not know what it does not know.
- It does not understand or intend its output as a person would.
How it works
| Text generator | Image generator | |
|---|---|---|
| Learned from | A very large amount of writing | Hundreds of millions of captioned pictures |
| Starts with | Your prompt | Random noise, and your prompt |
| Each step | Adds the next piece of text | Removes a little noise |
| Number of steps | One for each piece of text | A few dozen for the whole picture |
| Why results vary | Chance in choosing each piece | A different starting noise |
Try it yourself
Watch a sentence being written
You type: "Write one line about the monsoon." Follow the model as it builds the reply piece by piece.
Be the language model
Play with three or more people. You need paper and a pencil.
- Write the opening "Yesterday in the market I saw a" at the top of a page.
- The first player adds exactly one word that could sensibly come next, and passes the page on.
- Each player adds one word, reading only what is already written. Nobody may plan ahead or discuss.
- Stop after twenty words and read the sentence aloud.
- Start again from the same opening and see how different the second sentence is.
Think about it
- Did the sentence make sense, even though nobody planned it?
- Did it say anything true? How would you know?
Common mistakes
- Thinking the tool searches the internet and pastes what it finds. It generates from learned patterns.
- Thinking the same prompt should always give the same answer.
- Thinking that because the writing is smooth, the facts in it were checked.
Check yourself
Summary
- A generative model first learns patterns from a huge collection of examples.
- Text is generated one piece at a time; images are generated by removing noise step by step.
- It builds new content from patterns, with some randomness, and does not check for truth.
Finished reading?
Sources: Google: Introduction to generative AI · Elements of AI (University of Helsinki)