Definition
Why does your video feed look completely different from your friend's?
A recommendation system is an AI system that predicts which items a person is most likely to want, such as products, songs or videos, using what that person and many similar people have chosen before, and then shows those items first.
In simple words
A good shopkeeper remembers you. "You bought cricket gloves last month, so here are the new bats." A recommendation system is that shopkeeper, except it remembers millions of customers at once and notices that people who liked what you liked also liked something else.
Why it matters
Recommendations decide much of what people watch, hear and buy. If you know how a feed is chosen, you can steer it, and you can notice when it is keeping you scrolling longer than you meant to.
Key terms
- Recommendation system
- Software that predicts which items a person will want and shows those first.
- Collaborative filtering
- Recommending what people with similar tastes have chosen.
- Content-based filtering
- Recommending items similar to the ones you already liked.
- Engagement
- How much people interact with a product: watching, clicking, liking, sharing.
- Cold start
- The difficulty of recommending to a new user, or recommending a new item, when there is no history yet.
- Filter bubble
- The narrow view of the world a person gets when a feed keeps showing only what they already like.
Explained in detail
The problem of too much choice
A village shop has a few hundred items, and you can see them all. An online shop has millions. A music service has more songs than you could play in several lifetimes. Without help you would never find the few things you would love. A recommendation system is the answer to this problem. It reduces millions of items to the twenty that fit on your screen, and it does this separately for every user.
How similar people are found
The central idea, collaborative filtering, uses the behaviour of the crowd. Picture a huge table with a row for every user and a column for every film. Most cells are empty, because nobody has seen everything. The system finds users whose filled cells look like yours. If they loved a film you have not seen, it predicts you will too. It needs no knowledge of what the film is about. In 2006 a film-rental company offered a prize of one million dollars to anyone who could make its predictions ten per cent more accurate. Thousands of teams competed for three years, and the methods they developed are still in use.
The new user and the new song
Collaborative filtering fails when there is no history. A brand-new user has chosen nothing, and a song released this morning has no listeners. This is called the cold-start problem. Services handle it by asking a new user to pick a few favourites, by showing what is popular in their region, and by using the content of a new item, such as its language, singer and tempo, to guess who might like it.
What the system is trying to maximise
Every recommendation system is built to push some number upwards, and that choice shapes everything. A shop wants sales. A video app usually wants time spent watching, because more time means more advertisements shown. Content that provokes strong feelings, such as anger, shock or envy, tends to hold attention, so a system that chases watch time can drift towards showing more of it, even though nobody designed it to. Features such as autoplay and endless scrolling remove the natural moments at which you would stop.
Shopping has extra tricks
- Sponsored results: a seller has paid for a place at the top. Look for the small word "Sponsored" or "Ad".
- Changing prices: models adjust prices with demand, time of day and stock.
- False urgency: "Only 2 left!" and countdown timers are designed to stop you thinking.
- Fake reviews: some reviews are paid for or written by machines. Read the middling ones, not only the five-star ones.
Taking back control
You are not powerless. A recommendation system follows your signals, so you can steer it. Search for topics you want more of. Use "Not interested" firmly. Subscribe to channels you chose yourself, and visit them directly. Turn off autoplay. Clear or pause your watch history if your feed has drifted somewhere you dislike. And decide how long you want to spend before you open the app, because the app will never decide that for you.
How it works
| Method | The idea | Example |
|---|---|---|
| People like you | Find people with similar taste and suggest what they enjoyed | "Listeners of this singer also play…" |
| Items like this | Suggest things similar to what you already chose | After a cooking video, more cooking videos |
| Popular right now | Show what many people are choosing today | Trending songs for a new user |
The signals a feed watches
- How long you watch, and whether you watch to the end
- What you like, share, save or skip
- What you search for
- The time of day and the device you use
The system scores thousands of possible items, and the highest scores go to the top of your screen. Most feeds are built to increase the time you spend, because that is what the company measures. That goal is not the same as what is good for you.
Try it yourself
How a feed picks your next video
You have just finished a video about street food.
Match the signal to what the system learns
What does each action tell a recommendation system?
- Skipping a song after five seconds
- Buying a school bag
- Watching ten cricket videos in a row
- Tapping "Not interested"
Common mistakes
- Thinking a recommendation means the item is good or true. It only means you are likely to click it.
- Thinking the feed shows everyone the same thing.
- Forgetting that you can change your feed by changing your signals.
Check yourself
Summary
- Recommendation systems predict what you will want from your own choices and those of similar people.
- Watch time, likes, skips and searches are the signals that train a feed.
- A recommendation is a guess about your interest, not a judgement of quality or truth.
Finished reading?
Sources: Elements of AI (University of Helsinki) · Google: recommendation systems overview