Definition
What is actually inside a chatbot: a library, a brain, or something else?
A large language model (LLM) is a very large neural network trained on an enormous amount of text to predict the next piece of text. A chatbot is a product built around an LLM: it sends your conversation to the model and shows you the text the model produces.
In simple words
Imagine someone who has read more than any person could read in thousands of lifetimes, and who has become extraordinarily good at one game: guessing what words come next. They have never left the reading room and never checked a single fact for themselves. Ask them anything and they will continue the conversation fluently. Often they are right, because the right answer is what usually comes next. Sometimes they are confidently wrong.
Why it matters
Chat assistants are now used by hundreds of millions of people for study, work and advice. If you know what the model inside is and is not, you will understand why it is so capable, why it makes the particular mistakes it does, and how to use it sensibly.
Key terms
- Large language model (LLM)
- A very large neural network trained on text to predict and produce language.
- Parameter
- One of the adjustable numbers inside a model. Large models have billions.
- Token
- A small piece of text, such as a word or part of a word.
- Context window
- The amount of text a model can consider at one time.
- Knowledge cut-off
- The date after which a model has no training data, so it does not know later events.
- Pre-training
- The first, longest stage of training, in which the model learns language from vast amounts of text.
Explained in detail
Three words, taken one at a time
Large refers to size in two senses. The model has an enormous number of parameters, the adjustable numbers that training sets: hundreds of billions in the biggest models. And it was trained on an enormous amount of text, a large part of what has been published on the public internet along with many books. Language means it works with human language, and with other things that can be written down, such as computer programs and musical notation. Model means it is a mathematical system that represents patterns, not a store of documents.
How it is made
- Pre-training: the model reads text with the next piece hidden and tries to predict it, trillions of times. This takes weeks or months on thousands of specialised chips. At the end it can continue any text fluently.
- Fine-tuning: the model is trained further on examples of good conversations, so that it answers questions helpfully and does not simply continue your text.
- Feedback from people: human reviewers rate its answers, and it is adjusted towards the answers they prefer and away from harmful ones.
- Safety testing: teams try to make it misbehave, and add limits before it is released.
The letters GPT, which appear in the name of one well-known family of models, stand for Generative Pre-trained Transformer: it generates text, it was pre-trained on a great deal of it, and it uses the Transformer network design introduced in 2017.
What happens when you chat
The model itself has no memory between one reply and the next. The chat application creates the appearance of memory by sending the model the whole conversation so far, every time, together with hidden instructions from the company about how to behave. The model reads all of it and predicts what an assistant would say next. This is why a chatbot can refer to something you said ten messages ago, and why, in a very long conversation, it may lose track of the beginning: only a limited amount of text fits in its context window.
Four things that follow from this
| Fact about the model | What you will notice |
|---|---|
| It was trained up to a certain date | It does not know recent events unless it is connected to a search tool |
| It predicts likely text | It can state something false in a confident, fluent way |
| It works with pieces of text, not numbers or letters | It can slip on arithmetic, counting and spelling tasks |
| It learned from human writing | It reflects the knowledge, the styles and the biases of that writing |
Is it more than a word-guesser?
Predicting the next piece sounds trivial, but doing it well demands a great deal. To continue a physics explanation correctly, the model must have captured something about physics. To continue a story, it must keep track of characters and events. So these models show abilities nobody programmed directly: summarising, translating, reasoning through simple problems. Researchers disagree about how deep this goes. For a user, the practical position is clear. The model is highly capable and not reliable, and you cannot tell from the tone of an answer which of the two you are getting.
How it works
- You type a message.
- The application combines it with the earlier conversation and its own hidden instructions.
- The text is split into tokens and sent to the model.
- The model predicts the next token, adds it, and repeats until the reply is complete.
- The tokens are turned back into text and appear on your screen, often word by word.
Try it yourself
The journey of one chat message
You ask a chat assistant: "Why is the sky blue?"
True or false about language models?
Decide whether each statement is correct.
A model may not know about events after its training ended
A fluent answer is always a correct answer
A language model writes one piece of text at a time
The chat app sends the earlier conversation to the model each time
The model remembers you personally between separate chats
A language model looks every answer up in an encyclopedia
Common mistakes
- Thinking a chatbot is a search engine. It generates text; it does not retrieve verified pages unless built to.
- Thinking the model knows today's news.
- Typing private information into a chat as if it were a locked diary.
Check yourself
Summary
- An LLM is a very large neural network trained to predict the next piece of text.
- A chatbot sends your whole conversation to the model and shows what it generates.
- It is capable and fluent, limited by its training date, and not a checker of facts.
Finished reading?
Sources: Google: Introduction to generative AI · Elements of AI (University of Helsinki)