I5 · Machine Learning Foundations

Recommender Systems

A recommender system suggests items, such as films, products, songs or courses, that a user is likely to value. Content-based filtering recommends items similar to those the user liked before, using item features such as genre or ingredients. Collaborative filtering uses the behaviour of many users: user-based methods find people with similar tastes and recommend what they liked; item-based methods recommend items that are often liked by the same people. Matrix factorisation learns hidden "taste" factors for users and items from the ratings matrix. Hybrid systems combine these approaches.

12 min Intermediate · Sublevel Intermediate Jump to the activity

What you need: The earlier topics of this module.

Definition

You finish a cooking video and the app suggests three more that you actually want to watch. Behind that is a recommender system, comparing what you watched with what millions of others watched, and with what the videos are about.

A recommender system suggests items, such as films, products, songs or courses, that a user is likely to value. Content-based filtering recommends items similar to those the user liked before, using item features such as genre or ingredients. Collaborative filtering uses the behaviour of many users: user-based methods find people with similar tastes and recommend what they liked; item-based methods recommend items that are often liked by the same people. Matrix factorisation learns hidden "taste" factors for users and items from the ratings matrix. Hybrid systems combine these approaches.

In simple words

Two ways to get a film suggestion: ask a shopkeeper for "more action films like the one I loved" (content-based), or ask a friend whose taste matches yours what they enjoyed recently (collaborative). Recommender systems do both, at a huge scale.

Overview

  • Recommender systems suggest items a user is likely to value.
  • They learn from interactions (ratings, clicks, purchases) and from item and user features.
  • The user-item matrix is mostly empty, because each user has interacted with few items.
  • Cold start, for new users and new items, is a central challenge.
  • Evaluation uses hidden interactions offline (precision@k) and A/B tests online.
  • Good systems add diversity and avoid trapping users in filter bubbles.

Why it matters

Recommenders shape what billions of people see, buy, read and listen to every day, and they drive a large share of sales on shopping and streaming platforms. Understanding them helps you build one, judge its effects such as filter bubbles and popularity bias, and use apps more thoughtfully.

Key terms

User-item matrix
A table of users by items holding ratings or interactions; most cells are empty.
Content-based filtering
Recommending items whose features resemble those the user liked.
Collaborative filtering
Recommending from the behaviour of users with similar tastes.
Cosine similarity
A similarity between two vectors based on the angle between them, from −1 to 1.
Cold start
The difficulty of recommending for new users or new items with no history.
Matrix factorisation
Learning short user and item vectors whose dot product predicts the rating.

Types

Recommender systems come in several types. They differ in what information they use.

1. Popularity-based recommendation

Recommending the most popular or trending items to everyone.

  • How it works: Count interactions per item over a time window and show the top items, perhaps per region or category.
  • Example: "Top 10 songs in India today".
  • When to use it: Use it as a baseline and for new users with no history.

2. Content-based filtering

Recommending items similar to ones the user liked, based on item features.

  • How it works: Build feature vectors for items (genres, keywords) and a profile for each user, then rank items by cosine similarity.
  • Example: Suggesting more cricket documentaries to someone who watched two.
  • When to use it: Use it for new items and when explanations such as "because you watched…" matter.

3. Collaborative filtering (user-based and item-based)

Recommending from patterns of many users' behaviour.

  • How it works: User-based finds similar users and recommends what they liked; item-based recommends items liked by the same people as items the user liked.
  • Example: Predicting 4.13 stars for 3 Idiots for Riya from similar users Meera and Aman.
  • When to use it: Use it when there is plenty of interaction data; it can find surprising, useful suggestions.

4. Matrix factorisation

Learning short vectors for users and items whose dot products predict ratings.

  • How it works: Methods such as SVD-style factorisation or alternating least squares fill in the empty cells of the user-item matrix; the Netflix Prize (2006–2009) made them famous.
  • Example: Learning 20 hidden "taste" factors for every viewer and film.
  • When to use it: Use it for large, sparse rating or click data.

5. Hybrid and deep-learning recommenders

Systems that combine several signals, often with neural networks.

  • How it works: Two-tower models embed users and items separately for fast retrieval; ranking models then score candidates using many features.
  • Example: Video and shopping platforms combining watch history, item content and context such as time of day.
  • When to use it: Use them at large scale, where both cold start and personal relevance must be handled.

Explained in detail

Content-based and collaborative filtering

Content-based filtering builds a profile of each item, such as genres, cast, keywords or price range, and a profile of each user from the items they liked. It recommends items whose profiles are closest to the user's, usually with cosine similarity. It works for brand-new items as soon as their features are known, and it is easy to explain ("because you watched a thriller"), but it rarely surprises users with something different. Collaborative filtering ignores item features and relies on patterns in the user-item matrix. If you and another viewer rated many films similarly, a film they loved and you have not seen is a good suggestion. Item-based collaborative filtering, used by Amazon since the early 2000s, compares items instead ("people who bought this also bought") and is stable because item similarities change slowly. In the code below, Riya's ratings are most similar to Meera's (0.85) and Aman's (0.73), so collaborative filtering predicts 4.13 stars for 3 Idiots; content-based filtering on genres cannot separate 3 Idiots from Drishyam, giving both 0.5, because each shares only the drama genre with the sports films she liked.

Approaches compared

ApproachUsesStrengthWeakness
PopularityOverall countsSimple baseline; works for new usersSame list for everyone
Content-basedItem featuresNew items; explainableNarrow, predictable suggestions
User-based collaborativeSimilar users' ratingsFinds surprising itemsCold start; slow with many users
Item-based collaborativeItems liked by the same peopleStable and scalableNew items have no history
Matrix factorisationLearned latent factorsAccurate on sparse dataFactors are hard to interpret

Matrix factorisation, evaluation and responsibility

The Netflix Prize (2006 to 2009) made matrix factorisation famous. Each user and each film is given a short vector of, say, 20 numbers, learned so that the dot product of a user's vector and a film's vector approximates the rating. The learned factors often line up with ideas such as "serious versus light" or "action versus romance", though no one labelled them. Recommenders are evaluated offline by hiding some known interactions and checking whether they appear in the top-k list (precision@k, recall@k), and online by A/B tests that measure clicks, purchases or watch time. Responsible design matters: optimising only for clicks can push sensational content, popular items can crowd out niche ones, and users can be trapped in a filter bubble, so good systems add diversity, explain suggestions and let users give feedback.

How it works

  1. Collect interactions: ratings, purchases, clicks or watch time.
  2. Start with a popularity baseline.
  3. Build content-based and item-based collaborative recommendations.
  4. Evaluate with hidden interactions and precision@k.
  5. Add diversity, handle cold start with content features, and A/B test.

The maths

Cosine similarity
User-based prediction
Matrix factorisation
Learned vectors for user u and item i

Riya has not seen film X. Two similar users have: Aman, with similarity 0.9 to Riya, rated X 5 stars; Kabir, with similarity 0.3, rated it 2 stars. The weighted prediction is (0.9 × 5 + 0.3 × 2) ÷ (0.9 + 0.3) = (4.5 + 0.6) ÷ 1.2 = 4.25 stars, so X is a good recommendation. To get such similarities, compare rating vectors: if Riya rated three common films (5, 3, 0) and Aman rated them (4, 2, 1), the cosine similarity is (20 + 6 + 0) ÷ (√34 × √21) = 26 ÷ 26.72 ≈ 0.97.

In code

recommend_films.py
import numpy as np
import pandas as pd

films = ["Dangal", "Lagaan", "3 Idiots", "Queen", "Andhadhun", "Drishyam"]
ratings = pd.DataFrame(
    [[5, 4, 0, 1, 0, 0],      # Riya (0 = not seen)
     [4, 5, 5, 0, 1, 2],      # Aman
     [1, 0, 2, 5, 4, 0],      # Kabir
     [5, 5, 4, 0, 0, 1],      # Meera
     [0, 1, 1, 4, 5, 5],      # Arjun
     [4, 0, 5, 2, 0, 1]],     # Sana
    index=["Riya", "Aman", "Kabir", "Meera", "Arjun", "Sana"], columns=films)

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

R = ratings.values.astype(float)
sim = pd.DataFrame([[cosine(R[i], R[j]) for j in range(len(R))] for i in range(len(R))], index=ratings.index, columns=ratings.index)
print("Users most like Riya:", sim["Riya"].drop("Riya").sort_values(ascending=False).round(2).head(3).to_dict())

user = "Riya"
others = sim[user].drop(user)
unseen = ratings.columns[ratings.loc[user] == 0]
scores = {}
for film in unseen:
    raters = ratings.loc[others.index, film] > 0
    w = others[raters]
    scores[film] = (w * ratings.loc[w.index, film]).sum() / w.abs().sum()
print("Predicted ratings for Riya:", {f: round(float(s), 2) for f, s in sorted(scores.items(), key=lambda kv: -kv[1])})

genres = pd.DataFrame({"sport": [1, 1, 0, 0, 0, 0], "comedy": [0, 0, 1, 1, 0, 0], "thriller": [0, 0, 0, 0, 1, 1], "drama": [1, 1, 1, 1, 0, 1]}, index=films)
liked = genres.loc[ratings.loc[user] >= 4].mean()
content = {f: round(float(cosine(liked.values, genres.loc[f].values)), 2) for f in unseen}
print("Content-based match for Riya:", content)

Output

Users most like Riya: {'Meera': 0.85, 'Aman': 0.73, 'Sana': 0.5}
Predicted ratings for Riya: {'3 Idiots': 4.13, 'Andhadhun': 2.15, 'Drishyam': 1.6}
Content-based match for Riya: {'3 Idiots': 0.5, 'Andhadhun': 0.0, 'Drishyam': 0.5}

Try it yourself

Recommending courses on a learning app

A learning platform wants to suggest the next course to each learner.

Match the approach

Which approach does each description fit?

  • "Because you watched a cricket film"
  • "People who bought this also bought"
  • Hidden taste vectors learned from ratings
  • "Top 10 in India today"

Common mistakes

  • Evaluating on interactions the model was trained on.
  • Ignoring cold start for new users and new items.
  • Optimising only for clicks, which can promote sensational content and filter bubbles.

Check yourself

Question 1

1. Which approach can recommend a brand-new film on its first day?

Question 2

2. Neighbours with similarity 0.8 and 0.2 rated a film 5 and 1. What is the weighted prediction?

Question 3

3. What is the cold-start problem?

Summary

  • Recommenders suggest items using item features (content-based) or behaviour patterns (collaborative).
  • Cosine similarity and weighted averages power simple collaborative filtering; matrix factorisation learns hidden tastes.
  • Handle cold start, evaluate on hidden data, and design for diversity and user wellbeing.

Finished reading?

Sources: Google for Developers: recommendation systems course · Koren, Bell and Volinsky (2009), Matrix Factorization Techniques for Recommender Systems, IEEE Computer